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Foreword 



The papers contained in this volume were presented at the workshop Generalized 
Quantifiers and Computation of ESSLLI’97, the 9th European Summer School 
in Logic, Language and Information in Aix-en-Provence, France, August 11-22, 
1997. 

ESSLLI’97 was organised under the auspices of the European Association for 
Logic, Language and Information (FoLLI). The annual ESSLLI Summer School 
has developed into an important meeting place and forum for discussion for rese- 
archers and students interested in the interdisciplinary study of Logic, Computer 
Science, Linguistics, and Cognitive Science. 

The purpose of the workshop Generalized Quantifiers and Computation in 
ESSLLI’97 was to bring together experienced researchers and students in the 
areas of generalized quantifiers in finite model theory on one hand, and regular 
languages and circuit complexity on the other hand. 

The speakers at the workshop were: 

1. Anuj Dawar: Finite variable equivalence and generalized quantifiers. 

2. Kousha Etessami: Dynamic tree-isomorphism via first-order update. 

3. Lauri Hella: Enhancing fixed point logic with generalized quantifier. 

4. Christian Michaux: Definability and undefinability in extensions of Presbur- 
ger arithmetic, Buchi arithmetics. Applications. 

5. Ari Koponen: Definability of group theoretical notions. 

6. Kerkko Luosto: Ramsey theory is needed for solving definability problems 
of generalized quantifiers. 

7. Johann Makowsky: Invariant definability and circuit complexity. 

8. Juha Nurmonen: Local properties and finite structures. 

9. Iain Stewart: Logical characterization of oracle complexity classes. 

10. Jouko Vaananen: Generalized quantifiers and computation - an introduction. 

11. Helmut Veith: Generalized quantifiers in logic programming and query lan- 
guages. 

12. Heribert Vollmer: Generalized quantifiers in complexity theory. 

I am grateful to all the authors who contributed a paper to this proceedings 
volume. 
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Abstract. We review recent work in the field of generalized quantifiers 
on finite models. We give an idea of the methods that are available in 
this area. Main emphasis is on definability issues, such as whether there 
is a logic for the PTIME properties of unordered finite models. 



1 Introduction 

The ordinary quantifiers “for some” and “for all” are not sufficient for expres- 
sing some basic mathematical concepts. This led Mostowski to introduce in 
1957 generalized quantifiers, such as “for infinitely many” and “for uncountably 
many”. In the 1960’s and 1970’s these and other similar quantifiers were inten- 
sively studied by logicians. In this decade generalized quantifiers re-emerged in 
the framework of finite structures. Researchers in descriptive complexity theory 
and natural language semantics were looking at ways to formalize expressions 
like “for at least half” and “for an even number”. It turned out that the con- 
cept of generalized quantifier introduced by Mostowski, and further extended by 
Lindstrbm j2I], was the right answer. 

The goal of descriptive complexity theory is to classify problems, not ac- 
cording to how much resources they need when solved by means of a Turing 
machine, but according to how powerful logical languages are necessary for de- 
scribing the problems. For a quick example, let us consider some graph problems, 
say the problem whether a graph is triangle-free. 

We are given a graph, that is, a pair (G, E), where G is a set and E is a. binary 
predicate representing the edge-relation. Even before we think about triangle- 
freeness, we may ask how difficult it is to express the fact that (G, E) is indeed 
a graph. We have to check that E is symmetric and anti-reflexive. These two 
axioms can be easily expressed in a logical language as follows: In what follows, 
X and y are variables for elements of the domain G and E{x,y) stands for the 
assertion that x and y are in the edge-relation. Using standard logical notation, 
V means “for all”, — >■ means “implies”, -• means “not” and A means “and”, the 
following sentence expresses the fact that (G,E) is a graph: 

'ix'iy{E{x, y) — )> E(y, x)) A 'dx-'E{x, x).tp (1) 

* Reprinted with permission from the Bulletin of the European Association for Theo- 
retical Computer Science, 62 (1997), 115-136. 

J. Vaananen (Ed.): ESSLLP97, LNCS 1754, pp. 1-[T3 1999- 
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Likewise, the sentence 

Mx\/yMz^{E{x, y) A E{y, z) A E{z, x)). (2) 

expresses triangle-freeness of the graph (G, E) . From the point of view of de- 
scriptive complexity theory we may now say that to express the graph-axioms, 
and also the triangle-freeness of a graph, we do not need more than a logical 
language which has variables for elements and permits the use of V, A and 
— The smallest such language is called first order logic FO. 

The logical language FO has become an extremely successful tool in logic in 
this century. However, this success is almost solely due to its handiness in ex- 
pressing completeness properties of infinite structures such as “for every number 
there is a bigger number” or “between any two distinct numbers there is another 
number” etc. Problems arising in computer science do not usually involve struc- 
tures with universal completeness properties, not least because these structures 
(databases, computations, etc) are finite. Respectively, all the usual methods in 
the theoretical study of FO almost systematically fail in the finite context (see 
e.g. [mS]). Indeed, first order logic is not at all handy in expressing interesting 
properties of finite structures. Let us take the path-problem as an example. From 
a programming point of view it is natural to express the existence of a path from 
X to y in a graph (G, E) by a formula Conn(x, y) such that 

Conn(x, y) O [E{x, y) V 3z{E{x, z) A Conn( 2 :, j/))]. (3) 

This sentence features O (“if and only if”), 3 (“there exists”) and V (“or”), 
which are all definable in terms of the operations of FO. But more interestingly, 
it seems that in order to get the formula Conn(x,?/) we have to “solve” the 
equivalence m since Conn occurs on both sides. Such solutions are called fixed 
points. In general there may be several different fixed points, but there is a simple 
way of guaranteeing that a unique least fixed point exists. In {3D the existence 
of a unique least fixed point characteristically follows from the fact that on the 
right hand side the predicate Conn occurs only positively. The smallest logic 
extending first order logic where such fixed points can be expressed is called 
fixed point logic FP. It was introduced by Aho and Ullman PP in 1979. They 
also proved that the least fixed point of the equivalence is not first order 
definable. 

Fixed point logic is definitely very different from first order logic. Expressions 
of first order logic can be written down on a piece of paper, but how to write down 
the expression Conn? If we make a try, changing variables to avoid confusion, 
the result is likely to look like this: Conn(x, y) O 

E{x,y)V 

3zi{E{x, zi) A {E{zi,y)\/ 

3z2{E{zi,Z2) a {E{z2,y) V . . . ))). 

This sentence is never-ending! The expressions of fixed point logic are not senten- 
ces in the ordinary sense of the word but some kind of self-referential recursive 
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generalized sentences. Despite this difficulty in construing fixed point queries as 
sentences, fixed point logic has a very clear computational content. Whenever a 
graph and a fixed point expression is given, it is immediate how to check whether 
the graph satisfies the expression, and this can be done in polynomial time in 
the size of the graph. 

It would be tempting to conjecture that not only is every fixed point query 
in polynomial time, but conversely every polynomial time graph property is 
expressible as a fixed point query. Indeed, Immerman m and Vardi [21] 
showed in 1982 that this is true in the special case that the graph is endowed 
with a linear ordering of the vertices. In such a case it is possible to use tuples of 
vertices to build a model of a Turing machine inside the graph and imitate the 
polynomial time property by a suitable fixed point sentence. So in the presence 
of an ordering the fixed point approach is very powerful. However, what if we 
do not have an ordering of the vertices of the graph? 

It is relatively easy to see with present-day game-theoretic techniques that 
the polynomial time query “the number of vertices is even” is not expressible in 
fixed point logic on unordered graphs. The same is true of aZZnon-trivial counting 
queries. This observation has led to the following problem: 

Is there some natural extension of fixed point lo- (4) 

gic which expresses exactly the polynomial time 
queries on unordered graphs? 

raised first by Chandra and Harel |3]. 

If it were to be the case that P = NP, then such a natural extension would 
exist, namely existential second order logic defined below. First let us observe 
that first order logic FO makes sense in a framework that is much more general 
than just graphs. It is customary in logic to consider structures of a very general 
type, such as ordered structures, directed graphs, hypergraphs, groups, fields, 
etc. Common to all these structures is that there is one domain and one or more 
relations (functions can be treated as relations) and constants on this domain. 
Codd m defined on such finite structures the so called relation algebra as a 
kind of minimal database query language. Relation algebra is essentially the 
same thing as FO. More exactly, a structure consists of a set A, a sequence of 
relations on A, and a sequence of distinguished constants on A. Each relation is 
a subset of some Cartesian product of A. The number k is called the arity of 
the relation. First order logic for such abstract structures has a name for each 
relation and constant of the structure. (We use the same symbol for an object 
and its name, whenever no confusion arises.) The names of the relations and 
constant of a structure is called the vocabulary of the structure. 

For example, if (G, E) is a graph and is a subset of G, we can form a new 
structure (G,E,X) which has one relation E of arity 2 and one relation X of 
arity 1. We can say in first order logic that X contains neighbors of its elements 
(i.e. X is a union of connected components): 



yxyy{{X{x) A E{x, y)) X{y)). 
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Here X{t) is interpreted as is in X”. In existential second-order logic Xj we 
can form expressions such as 

3X{X{x) A ~^X{y) A VzVu((X(z) A E{z, u)) X(u))). (5) 

Here 3X is a so called second order quantifier because it binds a relation variable 
X rather than an element variable like 3z in ©■ Note that © says that there 
is no path from x to y. Thus it is equivalent to -iConn(cc, y). This shows that 
E\ can express things which are not first order definable. 

No-one knows whether is closed under negation or not in the framework 
of finite models. In infinite models infinity itself is a El concept. Its complement 
- finiteness - is not El, as follows easily from the so called Compactness Theorem 
of infinite model theory. 

Fagin proved that a query is NP if and only if it is expressible in El, in 
symbols NP = El. The role of guessing, inherent in non-deterministic computa- 
tions, is played by existential second order quantifiers. Fagin’s result holds on all 
structures, ordered or not, because if an order was not present, we could guess 
an ordering and continue as if the ordering was present. (Recall that ordering is 
used to simulate a Turing machine inside a structure). 

It is possible to express every fixed point query in El. Thus if P = NP, 
we have the extension El of FP which expresses exactly the polynomial time 
queries on graphs or indeed on any structures. The message is, that if we were 
able to answer question m in the negative, we would have proved P NP. 

The point of generalized quantifiers (to be defined below) is that they provide 
a very general yet coherent and mathematically exact approach to extending 
FO and FP. Proof techniques developed for them provide one possible road to 
analyzing question 0 and other open questions of descriptive complexity theory. 

2 Generalized Quantifiers — Definition 

We pointed out above, that there is no fixed point expression (and no first order 
expression) which would say that the number of vertices of a graph is even. 
Neither can we say in FP (or FO) that the degree of a vertex is even, or that 
at last half of the vertices have degree > 3, or that two vertices have the same 
degree, etc. There is an endless list of examples of simple properties which cannot 
be captured by FP. This motivates the following idea: We extend FO (and FP) 
by allowing a new operation Qevena :(. . .x . . .) with the interpretation 

Qeven 2 :(. . . X . . . ) <t4> the number of x with . . .x . . . is even. (6) 

Now the expression QevenXi?(x, y) says in a graph that the vertex y has even 
degree. By adding another new operation 

Qhaifx(. . . X . . . ) <t4> at least half of all elements x satisfy ... x ... , 

we can say things like at least half of the vertices of a graph have degree > 3: 

Qi,^itx3y3z3u{E{x, y) A E{x, z) A E{x, u)Ay^zAy^uAz^u). 
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The operations Qeven and Qhait are examples of generalized quantifiers. The 
extension of FO by Qe^n is denoted by FO(Qe™„) and the extension by Qhaif is 
denoted by FO(Qhait)- It is easy to verify that all queries that can be expres- 
sed in FO(Qeven) Or FO(Qhaif) are polynomial time computable, but not every 
polynomial time computable query is expressible in FO(Qeven) or FO(Qhaif)- 
There is an element of arbitrariness in the definition of Qeven and Qhaif above. 
One gets the feeling that something was needed and it was just thrown in ad hoc. 
The point is that generalized quantifiers provide a way of extending a language 
in a minimal way. Any extension of FO in which we can say one way or other 
that a predicate is satisfied by an even number of elements, and which satisfies 
some natural regularity properties, actually contains FO(Qeven)- 

Let us consider Hamiltonicity of a graph as an example. The probability 
that a randomly chosen finite graph is Hamiltonian tends to 1 when the size of 
the graph increases. There are logics, like FO and FP for which a zero-one law 
holds, that is, whatever sentence of the logic we consider, the probability that 
a randomly chosen finite structure satisfies that sentence tends to 0 or 1 when 
the size of the graph increases (see m for an informal discussion of this). Since 
Hamiltonicity cannot be expressed either in FO or in FP, the question was raised 
whether there is some extension of these logics in which Hamiltonicity can be 
expressed and which has a zero-one law. Dawar and Gradel 0 proved that the 
extension of FO obtained by adding the generalized quantifier 

QaB.taXy{. ■ - X .. .y ...) <^ the graph with the edge-relation , , 

{{x,y) : ... a; ... y ... } is Hamiltonian ' 

does not have the zero-one law. Thus no extension of FO capable of expressing 
Hamiltonicity can have a zero-one law. 

Definition 1 ( |21j ). Suppose L is a vocabulary. Any collection Q of structu- 
res of vocabulary L, that is closed under isomorphisms, is called a generalized 
quantifier of vocabulary L. 

The concept of generalized quantifier seems to be extremely general, and does 
not appear to have anything to do with the ordinary quantifiers V and 3. 

Let us discuss generality first. The important part of the definition is closure 
under isomorphisms. To see what this means, and how it limits generality, sup- 
pose L consists of one predicate symbol R which is unary i.e. of arity one. Up to 
isomorphism there are just n-\- 1 non-isomorphic L-structures of size n, namely 
for each m < n the structure Aff, with {!,... ,n} as universe and {!,... ,to} 
as the interpretation of R. We can picture these structures as a pyramid (see 
Figure [TJ. On vertical rows the size of the model is constant. On rows parallel 
to the left side of the pyramid the size of the predicate R is constant. On rows 
parallel to the right side the size of the complement of R is constant. 

Now a generalized quantifier of vocabulary L is essentially just a subset of 
this pyramid. Thus we have a pretty good picture of what kind of generalized 
quantifiers of vocabulary L there are. For more general vocabularies the picture 
becomes less and less clear. 



6 



J. Vaananen 



A 



4 

0 



Ag 



Aj 

A? 



Ai 



A? 



Ai 



Ai 

A| 



Ai 



A 



4 

4 



Fig. 1. Models with one unary predicate 



The relation between generalized quantifiers and ordinary quantifiers beco- 
mes evident from the actual use of generalized quantifiers. Suppose Q is as in 
the above definition. For simplicity, assume L consists of a relation symbol R, 
which is binary, i.e. of arity two, and of X, which is unary, i.e. of arity one. Using 
the quantifier Q we can talk about pairs {x, y) of elements satisfying some con- 
dition . . . X . . .y . . . and of elements z satisfying another condition z The 

expression 

Qxyz{. ..x...y...){ — z — ) 

says in a structure A with universe A that the L-structure {A,R,X) with uni- 
verse A and 

R = {{x, y) £ A : . . .x . . .y . . . holds in A} 

X = {z £ A\ z holds in A}, 

is in the set Q. 

In this framework the ordinary quantifier 3 can be defined as a generalized 
quantifier of vocabulary {X}, X unary, as follows: 

3 = {{A,X):XCA,Xy^9}. 

Thus 3x{. . . X . . .) holds in a structure A if and only if {A, X) £ 3 where 
X = {x £ A\ . . .X . . . holds in A}. 

Similarly 

V={(A,X) :X = A}. 

Let us denote by FO(Q) the extension of first order logic by the generalized 
quantifier Q. There is no special difficulty in adding a generalized quantifier Q to 
FP, although some attention has to be paid to the details. (Technically speaking, 
we should speak about the so called inflationary fixed point logic, unless Q is a 
so called monotone quantifier, but we disregard this detail here.) We denote the 
result by FP(Q). 

Do generalized quantifiers lead to a solution of question (S)? The answer is 
“yes and no” , as we shall see below. However, it is interesting to note already here 
that on almost all structures PTIME = FP(Q 3 „ 3 „) by a recent result of Hella, 
Kolaitis and Luosto m- This means that there is a representation of PTIME 
queries in terms of FP(Qeven) which holds on a randomly chosen finite structure 
with a probability which tends to one as the size of the structure increases. Even 
such a weak representation would not be possible with FO or FP alone. 
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3 A Hierarchy of Generalized Quantifiers 

To evaluate the merits of the concept of generalized quantifier we have to look 
at the results that it gives rise to. For example, hierarchy results are in general 
rather rare in complexity theory, while strong methods exist for proving hierarchy 
results for generalized quantifiers. 

The hierarchies of generalized quantifiers are based on counting how many 
variables and in how many formulas the quantifier binds. For example, the quan- 
tifier Qeven of ® binds one variable x in one formula. We say that it has type 
(1). The quantifier 



binds two variables (x and y) in two formulas, one variable per formula. We say 
that it has type (1,1). Finally Qnam of JZ|) binds two variables in one formula, 
we say that it has type (2). 

In general, a type is a finite sequence r = (ti, . . . , of positive integers with 
ti > • ■ • > tfc • A vocabulary of type r has an U-axy predicate symbol Ri for each 
i = 1, . . . ,k. A generalized quantifier has type r if its vocabulary is of type t. 

The type of a vocabulary determines how many models (up to isomorphism) 
there are of that vocabulary in each size of the domain. If the type r is unary, 
that is, = . . . = tfe = 1, then there are 



non-isomorphic models of vocabulary L of size n. If the vocabulary L is non- 
unary, that is, maxjti, . . . ,tk} > 2, then the number of non-isomorphic models 
of vocabulary L and of size n is harder to compute, but asymptotically it is (i) 



These formulas give immediately an idea of the number of quantifiers of a given 
type. The number per se is infinite, but we can get a finite number by considering 
the restrictions of the quantifier to a fixed finite domain A of cardinality n. If L is 
unary of type (ti, . . . , tk), the number of restrictions of quantifiers of vocabulary 
L to A is, of course 



lxy{. . . X . . . )( y ) there are as many x with . . . x . . . 

as there are y with y 



f n + 2^ -1 
2 ^= - 1 





and if L is non-unary, the number is 
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The point of calculating these numbers is that they give a simple counting 
method (invented by Per Lindstrdm) for proving results about generalized 
quantifiers. Suppose we want to construct a generalized quantifier Q of type 
(1,1) which is not definable in FO(Q') for any generalized quantifier Q' of type 
(1). Let us make a list 



of the possible sentences of FO(Q') that could give rise to a definition of such a Q. 
All we have to take care of is that in size n the quantifier Q is defined differently 
than what (j>n{,Q') says. But (roughly speaking) there are pos- 

sibilities for Q and only 2”+^ possibilities for Q' . So sheer counting shows that 
such a Q can be constructed. By elaborating this idea it is possible to prove a 
hierarchy theorem which demonstrates the existence of genuinely new quantifiers 
on each level of the type hierarchy (see below) . 

If r = (ti, ... ,tk) and r' = (t'^, . . . ,t),,) are types, we let r < r' if r precedes 
t' in the lexicographic order, that is, U < t' for the least i such that ti yf t' (or 
k < k' \i ti = t' for alH = 1, . . . , fc). Thus 

(1) < (1, 1) < . . . < (2) < (2, 1) < (2, 1, 1) < . . . < (2, 2) < . . . (3) < . . . . 

The order- type of this ordering is the infinite ordinal number . 

Theorem 1 (The Hierarchy Theorem [11 7] 1 . For every type t = (ti, . . . ,tfc) 
there is a generalized quantifier Q of type r such that Q is not definable in FO(Q') 
for any Q' of type < t. 

In loose terms: we get something genuinely new on every level of types. This 
result permits many refinements and variations. The most important of them 
says that Q can be chosen to be LOGSPACE, at least if = . . . = tk- Various 
other constraints can be imposed on Q. An essential feature of this result is that 
it is a pure existence result, reminiscent of Cantor’s diagonalization method. We 
may force the abstract object Q to satisfy some nice properties but we cannot 
“put our finger” on Q. 



4 Games and Quantifiers 

The Hierarchy Theorem of the previous section establishes the richness of the 
family of all generalized quantifiers. To study properties of individual quantifiers 
more refined methods are needed. Before introducing the main technical tool 
in the study of generalized quantifiers, the method of games, let us discuss the 
general problem of separating models by logical means. 

Suppose A and B are two models of the same vocabulary. We may assume 
they are both graphs, or we may assume they are two databases (of similar 
type). Picture [2] shows a simple example of the case that A and B are graphs. 
Graph B is connected while A is not, so the graphs are not isomorphic. What 
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Fig. 2. Two graphs 



is the simplest way of describing the difference between A and B? Certainly the 
difference can be described in FO with the sentence 

3x\3x2 ■ ■ ■ 3xi2(E{x\,X2) a E(x2,Xz) a ... a E{xix,Xi2) a E{x\2, Xi)) 

but this raises the new question, whether we can do the same with a sentence 
which would work for larger cycles, too. 

On the other hand, the sentence (recall ([S|)) 

3x3y{x yf y A 3X{X{x) A ~'X{y) A \fziu{{X{z) A E{z, u)) — >■ A(u))) (8) 

is a sentence of X} which is true in A and false in B. To see that this sentence 
is true in A, choose x from the left-hand cycle, y from the right-hand cycle, 
and let X be the whole left-hand cycle. Note that (|S]) works regardless of the 
sizes of A and B as long as A consists of two cycles and B of one. So ([SI) looks 
like a good logical method to describe the difference between A and B. But m 
is unsatisfactory as it uses the “second order” quantifier 3X, and no PTIME 
algorithm is known for checking the truth of sentences involving 3X. 

Then there is the fixed point formula Conn(a::, y) of but it is (in a sense) 
infinitary and therefore unsatisfactory. Of course we can also take the generalized 
quantifier of type (2) 

QconnXy{. . . X . . . y . . .) the graph which has an edge between 

every x and y with . . . x . . . y . . . 
is connected. 

Then FO((5conn) can in a trivial way describe a difference between A and B. 
The concept of generalized quantifier provides one possible framework for asking 
questions like: is FO((5conn) really the simplest logic in which the difference of A 
and B of Picture |2]can be expressed? So what is simpler than FO(Qco„n)? What 
if we replace Qconn by a quantifier of simpler type? The types simpler than that 
of Q are the unary types (1), (1,1), ... 

Proposition 1 f [15j l. No generalized quantifier of a unary type can express 
connectivity of graphs. 
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The main idea behind the proof is the use of the following “bijective” game. 
The game BGk{A, B) is played on two models A and B. There are two players 
called I and II. Player II starts by choosing a bijection fi between the domain 
A of A and the domain B of B. (If there is no such /i, Player II loses.) Then 
Player I chooses an element ai of A. Now Player II chooses again a bijection /2 
and Player I chooses again an element 02 of A. The game continues like this for 
k moves. In the end we have a correspondence 



ai /i(ai) 

02 / 2 ( 02 ) 

. ( 9 ) 

Ofc /fc(afe). 



Player II is the winner if he has been able to play all the k moves and the 
produced correspondence © is a partial isomorphism, i.e., preserves structure. 
For example, if A and B are graphs, this means that there is an edge between ai 
and aj in A if and only if there is and edge between fi{ai) and fj{aj) in B (and 
ai = aj fi{ai) = fj{aj)). A winning strategy of Player II in BG3(A,B) 

for the graphs of Picture E can be described as follows. The choice of /i can be 
arbitrary (because every single element looks the same in both models: edge to 
the left and edge to the right). After Player I has chosen oi. Player II chooses 
/2 so that it maps any element at a distance d < 2 from a\ to an element at the 
same distance d from /i(ai) preserving orientation. Otherwise /2 can be quite 
arbitrary as long as it is a bijection. After Player I chooses 02, Player II chooses /a 
so that it maps oi, 02, and their immediate neighbors to /i(oi), 72(02), and their 
immediate neighbors, preserving again orientation. After Player I has chosen 03 
it is easy to see that the mapping ai 1— >■ fi{ai) preserves the edge-relation. 

Let Q]^ be the family of all unary generalized quantifiers and FO(Q^) the 
extension of FO by all quantifiers in Qj^. 



Theorem 2 (|14|). A sufficient condition for a property P of finite models to 
be undefinable in FO(Qa) is that for all numbers k there are models A and B 
such that 

(i) A has property P, 

(ii) B does not have property P, 

(Hi) Player II has a winning strategy in the game BGfe(A,B). 



This theorem reduces the logical question whether a property P is expressible 
in terms of propositional connectives and quantifiers of a very general albeit 
unary nature, to the purely mathematical question whether certain models with 
certain combinatorial properties exist. For example, graphs like in Picture E 
together with Theorem E provide a proof of Proposition E 

The proof of Theorem Eis lengthy but not particularly difficult. The suf- 
ficiency of the condition for the undefinability of P is based on the following 
idea: Let us assume the model A satisfies a sentence Qx{. . .x . . .), supposedly 
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defining P, where Q is a generalized quantifier of type (1). (In general, the de- 
fining sentence need not start with Q.) Furthermore, suppose Player II has a 
winning strategy in PG'fc(A,B) with k sufficiently large. The actual choice of k 
is based on an inspection of the formula . . .x . . .. This strategy advises Player 
II to choose some bijection /i. Since Qx{. . .x . . .) is true in A, the structure 
{A, X) is in Q, where A is the domain of A and X is the set of elements x of A 
which satisfy . . .x . . .. Let B be the domain of B and Y the set of elements x of 
B which satisfy the same condition . . . x . . . in B. Now comes the fundamental 
idea of the game BGk- We claim that 

/i:(A,A)^(B,r). (10) 

Suppose X € A. We have to prove a G X /i(o) S Y. Suppose therefore e.g. 
X G X. Thus X satisfies the condition . . .x . . . in A. Let us let Player I choose 
tti = a; in the game BGk{A, B). Because Player II is playing a winning strategy, 
we know that Oi satisfies the same basic relations of A as /i(ai) satisfies in B. 
By formulating the induction hypothesis appropriately, we actually know that 
tti satisfies the same “definable” relations in A as /i(ai) satisfies in B. Here 
“definable” means definability in a restricted sense, which however includes the 
relation . . .x . . .. Since we know that oi satisfies condition . . .a\ . . . in A, we 
can conclude that /i(ai) satisfies condition .../i(ai)... in B. In particular, 
/i(ai) G Y, as desired. 

Now that we know dini), we may draw from {A,X) G Q the conclusion 
{B, Y) G Q, which means, by definition, that the sentence Qx{. . .x . . .) is true 
in B. Thus Qx{. . .x . . .) cannot, after all, define the property P. 

Theorem E has been successfully used to show that various graph properties 
(e.g. planarity) are not expressible in terms of unary quantifiers |2,'l] . Such results 
can be seen as formalizations of the vague intuition that some properties of binary 
relations cannot be reduced to properties of cardinalities of definable sets. 

What is really remarkable about Theorem E is that it generalizes both to 
non-unary quantifiers and to extensions of fixed point logic by generalized quan- 
tifiers. Let us call a type r = (ti, . . . ,tk) r-ary if max{ti, . . . , tk} < r. A voca- 
bulary (and a generalized quantifier) is called r-ary if its type is r-ary. The word 
“binary” is generally used for 2-ary. 

The game HG'^(A, B), an r-ary version of HGfc(A,B), is defined like the 
game BGk{A, B) except that when I moves, he chooses an r-tuple {a},. . . , a”) 
rather than a single element a^. So after k moves we have the correspondence 



ai = 1,... ,r 

^ fk(ai),j = 1,... ,r 



( 11 ) 



and Player II wins if this is a partial isomorphism. This game is immensely more 
difficult to win for Player II than BGk{A, B) since already fi has to preserve all 
r-tuples. For graphs this means that Player II cannot even make the first move 
without losing unless A and B are isomorphic. 
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Let denote the family of all r-ary generalized quantifiers, and FO(Q^) 
the corresponding extension of FO. 

Theorem 3 ( [Il4| ). Theorem m holds for r-ary generalized quantifiers, that is, 
if Qi is replaced by and _BG'fc(A,B) by BG5J(A,B). 

By means of this criterion it is possible to show that the following “Ramsey- 
quantifier” of type (r) 

. . X . . . ) 3A(A large A Vx S . . x . . . )) (12) 

is not definable in FO(Q^_i)[T^. Here “large” can be anything reasonable, e.g. 
n/2 or log(n), where n is the size of the model. We have denoted an r-tuple 
{xi, . . . ,Xr) by the bold face x. [A]” means the set of all r-tuples of distinct 
elements of X. The Ramsey-quantifier is not known to be PTIME in general. 
Indeed, in many natural instances it is NP-complete. But there are other explicit 
r-ary quantifiers, even in FP, which are not definable in FO(Q^_j^). One example 
is the transitive closure quantifier on r-tuples (see below) [10]. An interesting 
conclusion from this is: 

Theorem 4 ( [I14| L Fixed point logic FP cannot be represented as the extension 
o/FO by finitely many generalized quantifiers. 

Namely, the arities of the finitely many quantifiers would have a common upper 
bound r, so such a representation would contradict the existence of FP queries, 
which are not definable in FO(Q,,). This is a strong manifestation of the inherent 
incapability of (finitely many) generalized quantifiers to express recursion. 

What about combining recursion and generalized quantifiers? We have al- 
ready remarked that any generalized quantifier can be added to fixed point logic 
FP. Let FP(Q,,) denote the extension of FP by all r-ary quantifiers. 

We can replace the logic FO(Q^) by the logic FP(Q^) in Theorem O if we 
simultaneously replace the game BGk{A, B) by a new game BPGl.{A, B), called 
the bijective pebble game, which we now define. This game is like RG))(A, B) but 
it is potentially infinitely long (although it does not make sense to play more 
than nf moves, where n is the size of A). Therefore it is even harder for Player II 
than BGJ.{A, B). To counterbalance the length there is a special mechanism to 
limit the size of the final correspondence (cf. ITT1) L There are k pebbles. While 
in RG))(A, B) Player I chooses some r-tuple, in the game BPG^(A, B) he puts 
pebbles on elements of an r-tuple. What is the difference? Eventually he runs 
out of pebbles! Then he can take back some pebbles that he has used already 
and reuse them. At some point Player I decides that the game has lasted long 
enough. Now we look at the correspondence generated by the elements that have 
a pebble on them in the final position. So we form the mapping 

< ^ /u«) 

< ^ hdO, 



( 13 ) 
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where , . . . , is the sequence of elements with a pebble on them at the end. 
If this is a partial isomorphism, then Player II has won. 

The challenge that BPG'l.{A, B) presents for Player II is that he should find 
a strategy which works no matter how long the game has been going on and 
no matter how long it will go on. For example, if A and B are as in Picture |2] 
Player II wins BPG\{A, B) if and only if A: = 1. 

Theorem 5 ([114)1. Theorem d holds for fixed point logie and r-ary genera- 
lized quantifier, that is, if FO{Qi) is replaeed by FP(Q^) and i?Gfc(A, B) by 
BPGUA,B). 

With this criterion it is possible to prove that certain properties of r-ary 
relations are not expressible in terms of (r— l)-ary generalized quantifiers even if 
taking least fixed points of formulas are allowed. The first result in this direction, 
due to Cai, Fiirer and Immerman [2], exhibited a LOGSPACE property of 
graphs that is not expressible in FP(Q]^). This demonstrates in a powerful way 
the impossibility of solving question (|3) by means of fixed points and quantifiers 
which merely count sizes of definable sets. A further result of Hella [T3] showed 
that for every r there are LOGSPACE properties of finite models that are not 
expressible in FP(Q^). Hence: 

Theorem 6 (1131). On unordered finite models, PTIME is not the extension 
of fixed point logie by finitely many generalized quantifiers. 

This shows that if we want to answer © affirmatively, we have to look beyond 
fixed point logic and finite collections of generalized quantifiers. 

The difference between the Hierarchy Theorem (Theorem |TJ and the results 
of this section is that the counting method of the Hierarchy Theorem simply 
gives the existence of a quantifier with certain properties, with no concern to 
whether the quantifier has any intuitive meaning. The more elaborate game- 
theoretic methods of this section make it possible to take a concrete meaningful 
quantifier, like the Ramsey-quantifier, and prove that it cannot be reduced to 
simpler quantifiers. 



5 Quantifier Schemata 

We have pointed out that PTIME can be expressed in logical formalism as fixed 
point logic, provided that we restrict ourselves to ordered models. We have also 
pointed out that there is an extension of fixed point logic by a single generalized 
quantifier, which captures PTIME on almost all unordered finite models. Finally, 
we have concluded that in the framework of all unordered finite models there 
is no extension of fixed point logic by finitely many generalized quantifiers that 
would give all of PTIME. 

Gan we capture PTIME by adding an infinite number of generalized quanti- 
fiers to fixed point logic? There is a trivial answer. We can take one new quantifier 
for each PTIME query and add the resulting infinitely many quantifiers to FO. 
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Surely we get PTIME, but we have not gained anything. It makes more sense 
to look at infinite collections of quantifiers arising from some effective process, 
and then ask, can we get all of PTIME. 

There are several ways in which a single generalized quantifier can give rise to 
an infinite sequence of quantifiers. Recall the Ramsey-quantifier Ram^ defined 
in We can think of it as arising from the much simpler quantifier 

Qfx{. . .X . . .) 3X (|X| > f{n) A Va; G X{. . . a; . . . )) , 

where / : N — >■ N and n is the size of the model. With this quantifier one can 
say (by choosing f{n) = [n/2j + 1) things like: 

“Most vertices have a green neighbor.” 

Let us denote Rarrf by Ram^{Qf) if “X large” in (fT^ is defined as |X| > /(n), 
where again n denotes the size of the model. RamT(Qf) is called a Ramsey lift 
oi Qf. So with the sentence 

Ramf {Q f)xyE{x ,y) 



we can say 

“Most vertices are neighbors of each other.” (14) 

For every choice of /, the quantifier Qy of type (1) gives rise to the infinite 
sequence of quantifiers Ram'" {Q f) of higher and higher type. It turns out that for 
non-trivial / the quantifier Ram'"'{Qf) cannot be defined in terms of quantifiers 
of smaller type even if fixed points are used. But we would not get all of PTIME 
even if we added all possible Ramsey lifts of quantifiers of type (1) [T^. Other 
lifts of the nature of the Ramsey lift have been considered, especially in the study 
of natural language semantics, where they are used to formalize expressions like 

Most boys in my class and most girls in your class 
have all dated each other. 



However, the most interesting lift from the point of view of descriptive complexity 
theory is the resumption or vectorization lift. In a few words, the vectorization 
of a quantifier Q says about tuples what Q itself says about elements. Thus the 
second vectorization of Qeven says “for an even number of pairs {x, y) we have 
. . .X . . .y . . .. In general, let Q be a quantifier of type r = (ti, . . . ,tk)- Then 
its m’th vectorization Res''^{Q) is a quantifier of type (mti,... ,mt}f). So the 
arity of the quantifier increases by a factor of m. A quantifier Q of type r is 
a class of models of vocabulary L, where L consists of a ti-ary relation symbol 
Ri{x\,... ,xti) for each i = 1, . . . , fc. To define Res™‘{Q) we use the vector 
notation z to denote a sequence (^i, . . . , Zm) of m variables. Let the language L' 
consist of an (mti)-ary relation symbol R’i{x \, . . . , Xmu) for each i = 1, . . . , fc. 
Then Res'"'^{Q) is the class of models (A, R[,. . . , i?),) of vocabulary L' for which 
the L-structure (A™, i?i, . . . ,Rk) is in Q, where 

R, = {((a},... ,ajj,... ,(ar,... ,a-)) g(A-)*A 
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For example, suppose Q is the type (2,1) quantifier which consists of struc- 
tures {A,E, U), where U Q A and (A,E) is a graph with a clique of the size of 
the set C/. So in a model with domain A the expression 

Qxyz{. ..x...y...){ — 2 ; — ) 

says that (A, {(a, 6) £ : ... a ... 6 ... }) is a graph with a clique with as many 

elements as there are a G A with a , while Res"^{Q) is of type {2m, ni) and 

Qxyz(...x...y...)(___z___) 

says in the same model that (A™, {(a, b) £ (A™)^ : ... a ... b ... }) is a graph 

with a clique with as many elements as there are sequences a £ A™ with a 

Note that if Q is PTIME, then so is every Res'^{Q), so the vectorization lift 
is computationally simpler than the Ramsey lift. We denote by FO(Q^“) the 
extension of FO by all the vectorizations Res"^{Q), m = 1, 2, . . ., of Q. 

As an important concrete example, consider the Transitive Closure quantifier 
of type (2,1,1): 

TC = {(A, E, X, y) : (A, E) is a graph, A C A, F C A 

and from every x G X there is a path in the graph 
to some y GY.} 



Theorem 7 (|20jb NLOGSPACE= FO{TC^‘^) on ordered models. 

It is interesting to note that before this result it was not even known whether 
NLOGSPACE is closed under complements. It is an open problem, whether there 
is some natural logic L such that NLOGSPACE= FO(L) holds on unordered 
models. 

The Alternating Transitive Closure quantifier ATG consists of models of the 
form (A, E, X, Y), where (A, E) is a graph, X C A, Y C A, and every Xq G X has 
a neighbor xi whose every neighbor X 2 has a neighbor X 3 whose every neighbor 
Xi has a neighbor X 5 ...etc... until we come to an element of Y. Immerman m 
proved that PTIME= FO(ATC'^‘^) on ordered finite models. This result has the 
following interesting version on all models, ordered or unordered: 

Theorem 8 (|g). FP = FO (ATC<“). 

So although no finite sequence of generalized quantifiers can capture all of FP 
on unordered models (Theorem 0), the infinite sequence ATC^‘^ is capable of 
the job. 

Generalized quantifiers are thus, after all, able to express recursive definitions, 
as soon as sufficient arities, i.e. sufficiently long tuples are available. Still, it 
remains an open problem, whether PTIME = FO(Q''“) for some Q on unordered 
models. But Dawar has proved the following interesting result, which shows that 
the approach of (vectorizations) of generalized quantifiers is at least as good as 
any other: 
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Theorem 9 (@). If question 0 has a positive answer (in an exact sense), 
then there is a generalized quantifier Q so that PTIME = FO(Q^“) on all finite 
models. 

The methods available at the moment in the study of generalized quantifiers can 
be effectively used to study definability questions concerning individual quanti- 
fiers and families of quantifiers with a bound on arities. Unfortunately the same 
methods become extremely hard when applied to quantifier schemata. 
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Abstract. We survey recent results on logics with counting and their lo- 
cal properties. We first consider game-theoretic characterizations of first- 
order logic and its counting extensions provided by unary generalized 
quantifiers. We then study Gaifman’s and Hanf’s locality theorems, their 
connection with game characterizations, and examples of their usage in 
proving expressivity bounds for first-order logic and its extensions. We 
review the abstract notions of Gaifman’s and Hanf’s locality, and show 
how they are related. We also consider a closely related bounded degree 
property, and demonstrate its usefulness in proving expressivity bounds. 
We discuss two applications. One deals with proving lower bounds for 
the complexity class TG'’. In particular, we use logical characterization 
of TG° and locality theorems for first-order with counting quantifiers to 
provide lower bounds. We then explain how the notions of locality are 
used in database theory to prove that extensions of relational calculus 
with aggregate functions and grouping still lack the power to express 
fixpoint computation. 



1 Introduction 

Finite model theory is an active area of research, mostly due to its connections to 
theoretical computer science, in particular, database theory [T] and complexity 
theory Several important complexity classes have nice logical characte- 

rizations. For instance, the star-free languages are exactly the ones definable in 
first-order logic FO, least fixpoint logic LFP captures PTIME, and partial fix- 
point logic PFP captures PSPACE on ordered finite structures (see m)- These 
logics also have their counterparts in the theory of database query languages. 
For example, relational calculus, that underlies majority of practical query lan- 
guages, has exactly the power of first-order logic. The language Datalog with 
negation corresponds to LFP, and the extension of relational calculus with while 
loops corresponds to PFP (see B)- 

Several counting properties have also been considered in complexity and da- 
tabase theory. We give a few examples here. A logic can be extended with a 
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linearly ordered second sort universe of numbers, and counting quantifiers ‘there 
are at least n elements.’ A very strong result in shows that least fixpoint logic 
with such second sort counting fails to capture the complexity class PTIME. In 
circuit complexity theory, counting modulo quantifiers have been studied. Allo- 
wing gates which count inputs modulo a constant p, for every p, in the definition 
of AC'’, one obtains the class ACC. Again, this class has a logical characteriza- 
tion, and the problem whether the containment ACC C NC^ is strict, is still 
open. In jS], it was shown that the class TC°, that extends AC° with threshold 
gates, can be captured on ordered structures by first-order logic with the second 
sort counting, and simple arithmetic predicates on numbers. 

In database theory, one often extends traditional first-order based languages 
with aggregate functions, such as summing up all values in a column in a relation, 
or finding the average value. One also extends those languages with grouping, 
that permits queries such as finding the average salary in each department. While 
the expressive power of relational calculus and many other query languages is 
well understood, much less is known about their aggregate extensions. 



These applications have motivated a systematic study of the expressive po- 
wer of counting properties from the viewpoint of finite model theory. In finite 
model theory, one normally uses games to prove expressivity bounds. For ex- 
ample, Ehrenfeucht-Frai sse games are used for first-order logic, and bijective 
Ehrenfeucht-Frai sse games are used for FO extended with unary generalized 
quantifiers. In Section |H] we review rules of such games for FO and some of its 
extensions with counting. 



Playing a game often involves a complicated combinatorial argument. Many 
results on the expressive power of first-order logic and its extensions with coun- 
ting, in particular, those obtained as an attempt to avoid game arguments, give 
us the intuition that these logics can express only local properties, and lack a 
mechanism for fixpoint computation. In Section U we review several results of 
this kind. We consider Gaifman’s theorem (18| for first-order logic, which shows 
that every first-order formula is equivalent to a local one, in the sense that only a 
small part of a structure is relevant for evaluating the query given by a formula. 
We also study modifications of Hanf ’s result [22] . In this approach one counts 
the number of isomorphism types of fixed radius neighborhoods of points. If 
the result of this counting satisfies certain criteria, then the structures conside- 
red are guaranteed to be elementary equivalent in a certain logic. This technique 
has been modified for first-order logic HZ], first-order logic with counting modulo 
quantifiers |41| and first-order logic extended by all unary generalized quantifiers 
m, for the case of finite structures. Proofs of applicability of Hanf ’s technique 
typically are not very difficult [1 7\^ 5|4n|42IJ . We will see some examples in Sec- 
tion [H 



The above results have motivated a study of general notions of locality 
We review this line of work in Section O We show that Gaifman’s theorem gives 
rise to two general notions, one for sentences and one for open formulas. We 
formulate an abstract notion of locality that captures Hanf’s condition, and 
study the relationship between the notions of locality. We also consider the 
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bounded degree property, which is implied by all other notions of locality, and is 
particularly well suited for proving expressivity bounds, especially for properties 
involving fixpoint computation. 

In Section we discuss applications of the concept of locality in complexity 
theory. In particular, we study the relationship between the circuit complexity 
class TC° and other complexity classes such as L and NL. In Section |7] we 
consider applications in database theory. We review results on expressive power 
of relational languages that resemble commercial languages such as SQL. We 
show that queries such as transitive closure, are inexpressible in a theoretical 
language that has the power of core SQL. 

2 Preliminaries 

A relational signature cr is a finite set of relation symbols {i?i, ..., Ri}, each of 
which has an arity pi > 0. We write (T„ for a extended with n new constant 
symbols. A tr-structure is A = {A, Ri, . . . , Ri), where A is a non-empty set and 
Ri C interprets Ri. If the universe A is finite, the structure A is called 
finite. Unless mentioned otherwise, all structures considered here are assumed 
to be finite. When the notation is clear from the context, we write Ri in place 
of Ri- The class of finite cr-structures is denoted by STRUCT [cr]. For instance, a 
graph A = (A, E) is a structure over a signature which consists of a single binary 
relation symbol U. If if is required to be symmetric, then A is an undirected 
graph. 

If A C A, by A [ W we mean the structure with universe X, where the 
interpretation of each Ri is restricted to X. An isomorphism f : A ^ B is 
a bijection A ^ B such that (oi,... , Op J € Ri holds in A if and only if 
(/(oi),... ,/(apJ) G Ri holds in B. If oi,... ,an € A and 6i,... ,6„ G B, 
we say that {(ai,bi) I 1 < i < n} is a partial isomorphism A — >■ iB if it is an 
isomorphism A [ {a\, . . . , a„} — >■ B \ {6i, . . . , 6„}. 

Every formula • ■ ■ i with free variables Xi, . . . , Xm defines a query 

which maps a cr-structure A to an m-ary relation q^{A) = {a G A™ | A \= '0(a)}. 
We denote the corresponding structure by tp[A] = {A,qjjj{A)). An m-ary query 
q is definable in a logic C if there is a formula of that logic such that 
for every A G STRUCT[cr] we have q{A) = {a G A™ | A \= </5(a)|. As an 
example, consider the transitive closure query TRCL. Suppose we are given a 
graph A = {A,E). Then TRCL consists of all pairs (a, 5) G A^ such that there 
is an U-path from a to b, that is, 

<iTRCL = {(«,&) SA^ 

This query is definable in many fixpoint logics, and in second-order logic, but 
we shall see that it is not definable in FO and its counting extensions. 

The Gaifman graph G(A) of a cr-structure A is the undirected graph (A,E) 
where (a, 6) G E if and only if there is a tuple t G Ri for some Ri G cr such 
that a, 6 G t. Note that if A is an undirected graph, then A = G{A). The 



3oi, ... G A such that a = a\ and b = and 
E{ai, Oi+i) for all i 
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degree of a point is its degree in the Gaifman graph. The distanee d{a,b) is the 
length of the shortest path from a to 6 in G(A). For a G A, its r -sphere is 
Sf-{a) = {b G A \ d{b,a) < r}. For an n-tuple t we define = Uq 

The r -neighborhood of a tuple t is the tT„-structure Nf‘{t) = {A t Sf^{t),t)- That 
is, it is the restriction of A to with t being n distinguished constants. 

For instance, if ^ is a chordless cycle of length at least 2r + 2 and a G A, 
then N;^{a) is the chordless path of length 2r + 1 with one distinguished point 
a. We denote the isomorphism type of by tp1^{t). We emphasize here 

that if h : a) — >■ N:^{b) is an isomorphism, where a = (oi,... , a„) and 

b = (6i, . . . , bn), then h{ai) = bi, . . . , /i(a„) = bn, as neighborhoods N:^{a) and 
Nr^ib) are cr„-structures. 

If the structure A is understood, we omit it from the notations when conve- 
nient. 



3 Logics and Games 

In this section we introduce the logics that are considered in this paper. These 
logics are first-order logic FO and its extensions with various generalized quanti- 
fiers. All logics we consider are closed under Boolean connectives and first-order 
quantification, and are regular in the sense of [II 21 HI j (we do not go into details 
in this survey). We also review game-theoretic characterizations for elementary 
equivalence in these logics, and show how the games can be used to prove ex- 
pressivity bounds. 

We use the standard definitions for formulas and semantics of first-order logic 
FO (see e.g. [E])- Equality is treated as a special relation symbol which is not 
a member of the signature. The quantifier rank of a formula ip is defined to be 
the depth of quantifier nesting in p, and is denoted by qr{p). 

The rules of the first-order Ehrenfeucht-Frdi sse game are as follows. There 
are two players, the spoiler and the duplicator. Two tr-structures A and B and the 
number of rounds, say n, are given. In each round the spoiler first selects a point 
of one of the structures and the duplicator selects a point of the other structure. 
Let oi, . . . , a„ and bi, . . . , be the points selected after the last round from A 
and B, respectively. The duplicator is declared the winner if {(oi, | 1 < * < n} 

is a partial isomorphism A ^ B; otherwise the spoiler wins. We say that a player 
has a winning strategy if he can guarantee a win, no matter how the other player 
plays. This game is interesting because of the following result. 

Theorem 1 (cf. |1 Sj). Let A,B G STRUCT[ct]. Then the duplicator has a win- 
ning strategy in the n-round Ehrenfeucht-Frdi sse game if and only if A and B 
agree on all first-order sentences of quantifier rank up to n. □ 

We use the notation A =po ^ if th® duplicator has a winning strategy in 
the n-round game on A and B. The above theorem can be used to provide the 
following tool for proving expressivity bounds. 
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Corollary 1 (cf. [I13J1. A class C C STRUCT[cr] is not definable in first-order 
logic if and only if for every n there are u -structures A & C and B ^ C such that 
the duplicator has a winning strategy in the n-round Ehrenfeucht-Frdi sse game 
on A and B. □ 

In other words, if for every n we can find one a-structure from the class C 
and another from the complement of C, such that the duplicator can maintain a 
partial isomorphism for n rounds, then C is not definable in FO. 

We now give some examples. Note that in those examples, we do not spell 
out every single detail of the game argument - this may require more space than 
this entire section. We shall offer much simpler proofs of the existence of winning 
strategies in the next section, after we have introduced the ideas of locality. 

First, one can use Ehrenfeucht-Frai sse games to show that connectivity of 
finite graphs cannot be expressed in first-order logic. Assume that connectivity 
is definable, and take as a counterexample, for each finite n, ^ to be a chordless 
cycle of length 2" and to be a disjoint union of two chordless cycles of length 
2"’. The duplicator’s strategy in the n-round Ehrenfeucht-Frai sse game is to 
preserve, in each round j < n, distances up to 2"“-^ . The only way the spoiler 
could win is to show that in B there are two points with no path between them, 
whereas in A there always is a path between any two points. But if the spoiler 
cannot build a path between two points in n rounds, it does not matter how far 
these points are, or whether there is a path between them at all. This informal 
reasoning can be formalized to show that the duplicator has a winning strategy 
in the n-round Ehrenfeucht-Frai sse game over A and B. This result holds also 
for ordered structures, see [13]. Note that this shows that the transitive closure 
query TRCL cannot be defined in FO. Indeed, assume that a first-order formula 
y) defines TRCL. Then '^x'iyip{x, y) would be a first-order sentence defining 
connectivity. 

Next, assume that we are given two distinguished points a and 5 of a graph 
A. Then we cannot define, in first-order logic, the property that a and b have 
equally many neighbors. As a counterexample we can take, for every n, a to be a 
point which has n -|- 2 neighbors and 6 to be a point with n -|- 1 neighbors in some 
graph A. Since there are only n rounds available, the spoiler cannot demonstrate 
that b has fewer neighbors. 



3.1 Unary Quantifiers 

A commonly used way to increase the expressive power of first-order logic is to 
extend it with generalized quantifiers, cf. mi48i . The basic idea is that we are 
given a class of structures, and we can check whether a substructure defined by 
a given family of formulas belongs to this class. In computational complexity 
theory, generalized quantifiers are often considered as oracles. In this section, 
however, we concentrate more on logical aspects of generalized quantifiers. We 
now review this method in detail in the case of unary generalized quantifiers. 

Let be a signature of k unary symbols. Suppose K. is an isomorphism 
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closed class of (T™“’^^-structures. Then FO(Qa:) is the extension of FO by a new 
formula formation rule: 

if ipi{xi,yi) is a formula of FO((5 k) for z = 1, . . . , /c, then 
Q)cxi,... ,Xk{ipi{xi,yi),... ,ipk{xk,yk)) is a formula of FO(Qk). 

The corresponding semantic rule is: 

A \= Qkxi . . .Xkii’i{xi,ai), . . . ,ipk{xk,ak)) 
iff {A,ipi[A,ai],... ,ipk[A,ak]) G /C, 
where 4>i[A, ai] = {a € A \ A\= ipi{a, fli)}. 

Here the tuple gives the interpretation for those free variables in 'ipi[xi,yi) 
which are not equal to Xi. The extension FO(Q) by a set Q of unary quantifiers is 
defined similarly by adding to FO the above rule for each Qjc G Q. The quantifier 
rank of a formula of FO(Q) is defined as usually by the maximum depth of 
nesting of quantifiers (counting both first-order and generalized quantifiers) . We 
also write FO(Q„) for FO extended by all (continuum many) unary quantifiers. 
Note that the same definition can be used with other ambient logics, not just FO. 
In particular, we shall use £oou;(Qu), the infinitary logic extended with unary 
generalized quantifiers. 

We now list some well-known examples. 

Example 1. (1) The existential quantifier 3 corresponds to the class of structures 
{(H, P) I 0 P C H}. Similarly, the universal quantifier V can be identified 
with the unary quantifier which is defined by the class {(H, P) | P = A}. 

(2) Counting quantifier 3-^ can be defined by the class {{A, P) \ card{P) > k}. 
Note that each 3-^ can be defined in first-order logic; however, this requires k 
quantifiers 3, and increases the quantifier rank by k. In contrast, the counting 
quantifier 3-^ increases the quantifier rank by one. We denote the set of all 
counting quantifiers 3-*^ by C, i.e., C = {3-*^ | ^ > 1}. 

(3) Counting modulo m quantifier Dm is defined by the class {(H, P) | card{P) = 
0 (mod m)}. An easy first-order Ehrenfeucht-Frai sse game argument shows 
that Dm is not definable in FO whenever m > 1. Thus FO(Pm) is strictly 
more expressive than FO. For instance, the sentence D 2 x{x = x) of FO(P 2 ) 
says that the number of points in a structure is even; it is well-known that 
this property is not definable in first-order logic alone. 

(4) Majority quantifier MAJ, which is defined by the class {(A, P) | card{P) > 
^card{A)}, is not definable in FO, either. For example, in FO(MAJ) we can 
say that there is a node in a graph A that is connected to at least half of the 
nodes of A: the defining sentence is 3yMA3xE{y,x). This quantifier is also 
interesting in connection with capturing complexity classes, see Section 

(5) Extending FO with Rescher (bigger cardinality) or Hartig (equicardinality) 
quantifiers also increases the expressive power. Rescher quantifier R is defi- 
ned by the class {(A, P, S') | card{P) < card{S)} and Hartig quantifier H 
by the class {(A, P, S) | card{P) = card{S)}. Thus, for instance, given two 
points a and b in an undirected graph, Rx, y{E{a, x), E{b, y)) says that a has 
at most as many neighbors as b. Similarly, Hx,y{E{a,x),E{b,y)) says that 
a and b have equally many neighbors. 
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For each Qjc, there is a natural Ehrenfeucht-Frai sse style game-theoretic 
characterization for elementary equivalence in FO(Qa:)- We formulate the rules 
of this game for the counting modulo m quantifier. The rules of the game are as 
for first-order Ehrenfeucht-Frai' sse game except that now the spoiler may also 
choose a subset from one of the structures, say X C A. The duplicator has to 
respond by choosing a subset of the other structure, Y C B, which has modulo m 
equal cardinality to the spoiler’s choice, that is, card{Y) = card{X) (mod to). 
The spoiler then challenges the duplicator’s choice by selecting a point from the 
duplicator’s structure, b G B, and the duplicator has to choose a point from the 
other structure, a G A, such that a G X ii and only ii b G Y. Again, if oi, . . . , a„ 
are the points chosen from A and , . . . ,bn are the points chosen from B during 
n rounds, the duplicator wins if and only if {{ai,bi) | 1 < f < n} is a partial 
isomorphism A ^ B. We call this game the counting modulo to Ehrenfeucht-Frai 
sse game. 

Note that the first-order Ehrenfeucht-Frai sse game can be seen as a spe- 
cial case of the counting modulo to Ehrenfeucht-Frai' sse game: the spoiler can 
choose the empty subset from one of the structures, and, in order to win, the 
duplicator has to respond with the empty set. The spoiler and the duplicator 
then choose their points from the complement of the empty set, that is, without 
any restrictions, just as in the regular Ehrenfeucht-Fra'i sse game. 

The following theorem shows that the counting modulo to Ehrenfeucht-Frai 
sse game indeed gives us the game-theoretic characterization we were looking for. 
We use the notation A=pQ(-^ ^B when the duplicator has a winning strategy in 
the n-round counting modulo m Ehrenfeucht-Frai sse game. 

Theorem 2 (see PI]). Let A,B G STRUCT[cr]. Then if and only 

if A and B agree on all FO(Z?m) sentences of quantifier rank up to n. □ 



Corollary 2. A class C C STRUCT[ct] is not definable in FO{Dm) if and only 
if for every n there are a -structures A G C and B ^ C such that the duplicator 
has a winning strategy in the n-round counting modulo m Ehrenfeucht-Frai sse 
game on A and B. 

The intuition behind this theorem is that if all subsets of A and B that the 
spoiler and the duplicator can use in the game look similar, and there are modulo 
TO equally many of them, then ¥0(0^) can distinguish between A and B no 
more than FO can. In the next section we give a precise formulation for this 
intuition. 

Theorem [2] can be used to show limits of expressive bounds of counting 
modulo quantifiers. For instance, we can show that connectivity of finite graphs 
is not definable in FO{Dm), for any to. The construction is similar to the first- 
order case, but we also have to require that there are modulo to equally many 
points in both structures. Then we can proceed very much like in the proof for 
first-order. Similarly, we can show that the majority quantifier MAJ (or Rescher 
or Hartig quantifiers) cannot be defined in FO(Z?m)- To see this, we can, for 
instance, take, for each n, A = (A, U) to be a set with a unary relation U 
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such that card{A) = 3nm and card{U) = 2nm. Take B = {B, U) which satisfies 
card{B) = 3nm but card(U) = nm. It is not difficult to show that the duplicator 
has a winning strategy in the n-round counting modulo m Ehrenfeucht-Frai' sse 
game over A and B. But obviously A |= MAJxl7(x) and B ^ MAJxC/(x). These 
results can be extended to the ordered case, see [4T] . 

If we want to give a game-theoretic method to prove expressive bounds for 
first-order logic with all unary quantifiers, different techniques must be used. 
The method we employ here is based on bijective Ehrenfeucht-Frdi sse games. 
The rules of the game are the following. As before, the players are the spoiler and 
the duplicator. There are two cr-structures A and B and the number of rounds, 
say n, given. In each round i the duplicator first selects a bijection fi'.A^B 
(if card{A) ^ card{B), then the duplicator loses), and then the spoiler selects a 
point Oi C A. The duplicator has to select the point fi{ai) from B. This continues 
for n rounds. After the last round, the duplicator is declared the winner if and 
only A {{ai, fi{ai)) \ 1 < * < n} is a partial isomorphism from A to B. We use 
the notation A if the duplicator has a winning strategy in the n-move 

bijective game on A and B. 

It turns out that this game characterizes elementary equivalence in a logic 
that is stronger than FO(Q„). This logic is obtained from FO(Q„) by allowing 
infinite disjunctions and conjunctions, but by keeping quantifier rank bounded. 
More precisely, let £oou;(Q„)^ be the extension of FO(Q„) where infinite dis- 
junctions and conjunctions are allowed but quantifier rank of each formula is at 
most k. The union of all these logics CaouiiQu)^ natural numbers k < uj 

is denoted by £oow(Q„)“ (that is, the depth of nesting of quantifiers in each 
formula is finite) . Methods used in give us the following result (a proof can 
be found in |M]j. 

Theorem 3 (see |23l24j l. Let A,B G STRUCT[cr]. Then A - B if and only 
if A and B agree on all Tooaj(Q«)“ sentences of quantifier rank up to n. □ 

Corollary 3. A class C C STRUCT[(t] is not definable in Tooi.j(Q«)“ if 
only if for euery n there are A G C and B ^ C such that the duplicator has a 
winning strategy in the n-round bijectiue Ehrenfeucht-Frdi sse game on A and 
B. □ 

Note that the expressive power of FO(Q„) is strictly weaker than that of 
>Coow(Qu)‘^- III FO(Qu)) if is not possible to express the second vectorization 
of Hartig quantifier (that is, the equicardinality quantifier for pairs) j.'ffl] . while 
techniques used in m show that every vectorization of every unary quantifier 
can be defined in Coouj (QJ It also follows from |2I] that £oouj(Qu)‘^ is as 
strong in expressive power as Toow(C)‘^. (In [SI], this was shown for finite- 
variable logics, but the same proof technique works for £oow(Qu)‘^ ES]-) 

Although £ooo;(Q„)“ has strong counting power, the game characterization 
can be used to show that connectivity and transitive closure are not definable in 
it. The idea is the same as before: For each n, we can take A to be a chordless 
cycle of length 2"+^ and B a disjoint union of two chordless cycles of length 2". 
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Now the duplicator’s strategy is to choose in round j a bijection that preserves 
the distances up to between the points next to the already chosen ones. 
A combinatorial proof can be given that shows the existence of such a strategy 
|27l : however, we shall see a much easier way to establish this in the next section. 

We remark that bijective games are useless in the presence of a linear order. 
In order to win, the duplicator has to follow the linear order when he chooses 
bijections (otherwise the spoiler wins in the next two rounds). Thus, there is 
essentially only one bijection the duplicator can choose. This gives us an example 
of difficulties that arise when one attempts to prove expressivity bounds in the 
ordered setting. In the subsequent sections, we shall face similar problems several 
times. 

Another interesting counting logic is FO + COUNT, cf. jbSISOj . This is a 
two-sorted logic with the second sort being the sort of natural numbers. More 
precisely, in this approach a structure is of the form 

A= ({ui,... ,u„}, {!,... l,max). 

Here relations Ri apply to the non- numerical domain {r)i,... ,Vn}, while the 
linear order <, the BIT predicate and the constants 1 and max (interpreted as 1 
and n) refer to the numerical domain {1, . . . , n}. Here BIT{i,j) holds if and only 
if the zth bit in the binary representation of j is one. These two disjoint domains 
are connected by allowing formulas of the form Bixip(x) with the semantics that 
at least i elements satisfy ip, i.e., card{{a G A \ A\= <p(a)}) > i. Here i refers to 
the numerical domain and x refers to the non-numerical domain; the quantifier 
3ix binds x but not i. As an example, consider the sentence 

3i3j[{j + j = i) A 3ixif{x) A yk(3kxip{x) -A k < i)]. 

This sentence tests if the cardinality of {a | <p{a)} is even. Indeed, 3ixip{x) A 
yk(3kxtp{x) -A k < i) holds iff exactly i elements satisfy p, and i is even since 
it is of the form 2j. Note that we used the fact that -|- and * are definable as 
ternary predicates in the presence of the BIT relation, cf. nasHi- 

Remark 1. L. Hella made the following observation. While first-order logic exten- 
ded by Rescher quantifier R is in general strictly weaker than FO -I- COUNT, in 
the presence of a built-out linear order, FO(i?) and FO -I- COUNT have the same 
expressive power. Clearly, Rx, y{(p{x),tp{y)) can be written in FO-bCOUNT. On 
the other hand, 3ixp{x) can be expressed by Rj,x{j < i,(p{x)). 

A game-theoretic characterization for elementary equivalence in FO-bCOUNT 
was introduced in m and used subsequently in m- However, we do not go into 
detail here, mainly due to the fact that the counting games of m are subsumed 
by the bijective games. The logic FO -b COUNT has a number of applications in 
computer science, in particular, in complexity theory. This will be discussed in 
Section E] 

Finally, we refer the reader to [dll48j for a more detailed overview of results 
on generalized quantifiers in finite-model theory. 
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4 Gaifman’s and Hanf’s Conditions 

The game-theoretic characterizations for elementary equivalence of logics con- 
sidered in the previous section gave us a vague intuition that these logics can 
only express local properties. This intuition will be formalized in this section. 
We review theorems by Gaifman and Hanf, and their modifications. 



4.1 Gaifman’s Theorem 

We start with Gaifman’s theorem m- Let .A be a a-structure. Recall that the 
distance d{a, b) is the length of the shortest path from a to 6 in the Gaifman graph 
of A, and Sr{a) = {b \ d{b, a) < r,a G a}. For each fixed fc, there are first-order 
formulas that define the relations d(a, b) > fc, d(a, b) = k and d{a, b) < k (see 
1131). Hence, bounded quantifications of the form Va; £ Sk{y) and 3a; £ Sk{y) 
are expressible in first-order logic for every fixed k. A formula {y) is called 
r -local around y if every quantifier in it is of the form Va; £ Sk{y) or 3a: £ Sk{y) 
with k < r. A sentence tp is called basic r -local if it is of the form 

3a;i...3a;m( f\ f\ d{xi,Xj)>2r) 

l<z<m 



where (x) is an r-local formula around x. 



Theorem 4 (Gaifman [18] L Every first-order sentence is equivalent to a Boo- 
lean combination of basic r-local sentences, and every first-order formula p>{xi, . . . 
. . . , Xn) is equivalent to a Boolean eombination of t -local formulas around Xi, . . . 

. . . ,Xn and basic r-local sentences. Furthermore, r < t < 

and m < n -\- qr{ip). □ 

Gaifman’s theorem tells us that every first-order formula can see only a bo- 
unded number of local neighborhoods in a structure, i.e., only a small part of the 
input. This is indeed a formalization of the informal statement that first-order 
logic can express only local properties. 



2t 2t 




Fig. 1. Formula p(x,y) cannot distinguish (a, t>) from (b,a). 



Example 2. We show that first-order logic cannot express the transitive closure 
of a directed graph. Assume, to the contrary, that there is a first-order for- 
mula ip{x,y) in the language of a single binary relation E, such that (p[A\ = 
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{A,{{a,b) G A \ A \= (f{a,b)}) is the transitive closure of A. Apply Gaifman’s 
theorem to it, and find t and r. Now consider the graph shown in Figure [I] It 
is a successor relation, on which we select two points, a and b. Assume that 
d{a, b) > 2t, and the distances from a and b to the start and the end node 
of the graph are at least t + 1. Then the t-neighborhoods of {a,b) and (b,a) 
are isomorphic, and by Gaifman’s theorem, ip cannot distinguish (a, 5) from 
(6, a). Thus, (fi cannot define the transitive closure, since (a, 6) G TRCL{A), but 
lb, a) ^ TRCL{A). 

4.2 Hanf’s Theorem and its Modifications 

While Gaifman’s theorem helps prove expressivity bounds for FO directly, with- 
out resorting to establishing a winning strategy for the duplicatoi0 Hanf’s theo- 
rem m and its numerous modifications I17I24I40I41) provide criteria for the 
existence of a strategy for the duplicator that is based on counting of small 
neighborhoods in two structures. 

Hanf’s theorem was originally proved for infinite structures. It was observed 
by Fagin, Stockmeyer and Vardi m that the technique can be modified to 
be applicable to finite structures. The extensions of Hanf’s technique [24140141] 
follow the ideas of [l7] . 

Let A be a tr-structure and a G A. Recall that the isomorphism type of 
is denoted by tp-^{a). Let r be an isomorphism type of a cri-structure 
(tj extended with one constant). We denote the number of points a G A whose 
d-neighborhoods realize r by nd{A,T). That is, 

nd{A,T) = card{{a G A\tp^{a) = t}). 

For example, if A is a chordless undirected cycle of length at least 2d -|- 2 then 
there is only one isomorphism type r of a d-neighborhood of a point occurring 
in A', the chordless path of length 2d + 1. In this case nd(A,T) = card{A). 

We call structures A and B {d,m)- equivalent if for every isomorphism type 
T they have exactly the same number of points whose d-neighborhoods realize 
T, or both structures have at least m such points, that is, 

min{nd{A, t) , m) = mm{nd{B,T),m). 

The modification of Hanf’s theorem for the finite case is the following. 

Theorem 5 (ini)- Let n and f be positive integers. There are positive integers 
d and m sueh that whenever A and B are (d, m)-equivalent struetures where every 
point has degree at most f, then A =Pq that is, A and B satisfy the same 
sentenees o/FO of quantifier rank up to n. □ 



^ New winning conditions for the duplicator based on Gaifman’s theorem were pre- 
sented recently in |47| . 
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Note that since we consider finite models, for any A there is a number / that 
exceeds degrees of all points of A. This leads to the following. We say that A 
and B are d- equivalent, written as A B, if for every type of a d-neighborhood 

of a point, r, equally many points realize it in A and B. That is, 

A^dB iff nd{A,T) = nd{B,T) for every r. 

Corollary 4 (HI])- Let n he a positive integer. There there exists a positive 
integer d such that A ^d B implies A =pQ B. □ 

This result makes precise the intuition that counting power of first-order 
logic is rather limited. It also shows that only local neighborhoods are relevant 
for elementary equivalence in FO. Most importantly, the result above yields 
much simpler proofs of expressivity bounds that those based on games. Below 
we give a canonical example of applicability of Hanf’s technique. 

Example 3. We show that connectivity of finite graphs is not definable in FO. 
Assume, to the contrary, that it is definable by a FO sentence <P of quantifier 
rank n. Apply Corollary U] to find d > 0 such that A ^d B would imply A\= 
iS B Now let A be a (chordless) cycle which has length 4d -|- 4, and let B 
be a disjoint union of two chordless cycles of length 2d -I- 2, see Figure 0 




Fig. 2. Hanf’s technique proves that connectivity is not first-order 



As we noticed before, there is only one type of a d-neighborhood that these 
graphs realize, namely a chain on 2d -I- 1 points. Thus, A ^d B, since they have 
the same number of vertices. At the same time, A is connected, but B is not, 
proving that connectivity cannot be expressed by T>. 
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Other examples, proved previously with games, can be shown to be derivable 
from Hanf’s technique. For example, with the concept of (d, 77i)-equivalence, it is 
now easy to show that the majority quantifier (or Rescher and Hartig quantifiers) 
are not definable in first-order logic. □ 

Before we outline the proof of Theorem [S] in we state Hanf’s theorem 
from [22j . It says that two arbitrary structures A and B are elementary equivalent 
if Sf^{a) and S^{b) are finite, for every finite r and every a G A and b G B, 
and, for each r > 0 and each type r of an r-neighborhood of a point, either 
nr{A, t) = rir{B, r) < to, or both A and B have infinitely many realizers of type 

T. 

Now we explain how Theorem |5]is proved. First, d is taken to be ^ , and 

m is taken to exceed the size of any d-neighborhood of a point in a structure 
whose degrees are bounded by /. Define dj = - — 2 ~^ for j < n; in particular, 
dj = S ■ dj_|_i -b 1, di = d and d„ = 1. It is then shown that the duplicator 
can play in the first-order Ehrenfeucht-Frai sse game in such a way that after 
each round j < n, ii a G and b G are the points chosen so far, then 
= tp^,{b). This suffices, since . . . , a„) = tpf{bi , ... , 6„) implies 

that the mapping ai i— > 6^ is a partial isomorphism. The condition that A and B 
are (d, m)-equivalent allows the duplicator to make the first move by choosing 
a point with a given type of its d-neighborhood. Suppose that the duplicator 
maintained the condition successfully for the first j rounds. That is, we have 
tpr^,{a) — tp^.{b). Assume that a G A is spoiler’s next choice in the game. If 
® ^ duplicator selects the point b G which is 

given by the isomorphism between and N^.{b). Then one easily checks 

that tp^.^^{aa) = tp^.^^{bb). If a ^ 5^.^ then the duplicator can choose 

any b ^ such that tp-^,^_^{a) = tp^,^^{b). If m is chosen as above, 

such a point b always exists. Then again we can see that tpr^,^_^(aa) = tp^^^^{bb) 
holds, since no points in (a) and belong to the same tuple of an 

A-relation, and likewise for B. 

Having established Hanf’s condition for FO, we turn to counting extensions 
of FO. In [H], the proof of Theorem Elwas modified for FO{Dk), k > 1. Two 
structures A and B are called (d, m, DA)-equivalent, if for each isomorphism type 
T they have the same number of points whose d-neighborhoods realize r, or in 
both structures there are at least m such points but modulo k equally many, 
that is, 

min(rid(A, r), m) = min(nd(S, r), m) and nd{A,r) = nd{B,r) (mod fc). 

Note that (d, m, Di)-equivalence is just the (d, m) -equivalence. Thus, the follo- 
wing result extends Theorem 0 

Theorem 6 ( 11411 1. Let n, f and k be positive integers. There are positive in- 
tegers d and m such that whenever A and B are (d, m, Dk)- equivalent structures 
where every point has degree at most f, then A=pQ^jy^^B . That is, A and B 
satisfy the same sentences ofFO{Dk) of quantifier rank up to n. □ 
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The intuition behind this theorem is that the counting power of FO{Dk) is 
rather limited, and that FO(Z3fe) is not much stronger than FO. For instance, 
we can use Theorem to show that connectivity is not definable in FO(Dfe) for 
any k, by using the same argument as in Example [S] 

The proof of Theorem in m follows the same idea than the proof of Theo- 
rem 0 One could ask whether this proof technique can be easily modified to 
prove elementary equivalence for FO(Q), where Q is an arbitrary unary quan- 
tifier. It turns out that in this technique additivity property of the quantifier 
is crucial, and all unary quantifiers which satisfy this property are essentially 
counting modulo quantifiers. However, we now show a more general result that 
describes a winning condition in a bijective game. 

Theorem 7 (| |24l40 |b Let n he a positive integer. There is a positive integer 
d such that A ^ implies A particular, if A then A and B 

satisfy the same sentences of CaouiQu) of quantifier rank up to n. □ 

Before we explain how this result is proved, we give the following alternative 
definition of d-equi valence. Two structures A and B are d-equivalent if there 
exists a bijection f : A ^ B such that for every a G A, 

tPd(a) = tp^ifia)). 



Now the proof of Theorem |3is very similar to the proof of Hanf’s theorem for 
first-order logic. We again let dj be - — and take d to be di. The duplica- 
tor’s strategy is to play so that after each round j in the bijective Ehrenfeucht- 
Frai sse game we have tp-ffa) = tp^fb), if a G A^ and b G are cho- 
sen during those j rounds. The first round bijection is given by the refor- 
mulation of d-equivalence above. Assume j rounds have been played, and we 
have tp-^,{a) = tp^,{b), where dj = 3 ■ dj+i -I- 1. Assume that tp^,^^{a) = r, 
for a G Then = r, and h{a) G (b), where 

h : iV4(a) ^ N^ib) is an isomorphism. Thus, the number of points reali- 
zing r is the same in complements of S:^dj+i+ii^) ^ 2 dj+i+ii^)’ hence 
we have a bijection g : A — B — with the property 

^P'dj+ii^) — ^P%+ii9i^))- define a bijection /j+i for the round j -I- 1 to 

be h on and g on its complement. It is routine to verify that fj+i 

is a bijection, and that tp^.^^(aa) = tp^.^^{bfj^i{a)) for every a G A. After the 
last round we have vectors with isomorphic 1-neighborhoods; hence they define 
a partial isomorphism. 

Theorem H shows the limits of expressive power provided by all unary quan- 
tifiers. It also significantly simplifies proofs of expressivity bounds, as applying 
bijective games is usually not a very easy task. For instance, we can use Exam- 
ple [3] to show that connectivity of finite graphs is not definable in Toooj(Qu)“i 
thus avoiding all the tedious combinatorics involved in an argument based on 
bijective games. 

As another example, we consider classes of undirected graphs which are closed 
under stretching. More precisely, let A be a graph and let Ad be the graph 
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obtained by replacing every edge of ^ by a path of length 2d + 1. Then each 
d-neighborhood of a point in Ad contains at most one node whose degree is 
greater than two. We say that a class C of graphs is closed under stretching if 
Ad & C for every A G C and for every positive integer d. Now it is easy to see 
that if A and B are 1-equivalent graphs, then Ad and Bd are d-equi valent. In 
other words, if there are the same number of points in both graphs A and B of 
each degree, then Ad and Bd are d-equivalent. It can then be proved that if a 
class C and its complement are closed under stretching, and there are Ag C and 
B as above, then C is not definable in £ooi.j(Q„)“ [2S]. This argument shows 
that graph properties such as planarity and 3-colorability are not definable in 
>Cooc(Q„)“. 

Hanf’s technique was also used by Etessami |T5] (although a preliminary 
conference version m had a proof based on counting games of m)- It was shown 
in m that a linear order cannot be defined in FO -I- COUNT from its underlying 
successor relation. The proof relies on Hanf’s technique for FO -I- COUNT. Its 
applicability follows from Theorem [71 since every FO -I- COUNT sentence is 
definable in £ooa;(Q„)“- 

Summing up, the combinatorial arguments in this section allow us to sim- 
plify many proofs that were originally shown by using difficult game-theoretic 
arguments. Furthermore, we can also prove, often quite easily, new nondefina- 
bility results. Note, however, that in the presence of a linear ordering, none of 
these results can be applied to derive inexpressibility results. This is because 
in the presence of an order, every point is a neighbor of any other point, and 
thus S^{a) contains all elements of A. Many results proving expressive bounds 
of these logics on ordered structures still use games (see [13| for first-order case 
and m for FO{Djn))- However, bijective Ehrenfeucht-Frai sse games cannot be 
applied at all in the ordered case. In fact, this is closely connected to some deep 
problems in circuit complexity, as will be explained in Section (El 



5 Abstract Notions of Locality 

On the surface, Gaifman’s theorem and various forms of Hanf’s theorem appear 
to be quite unrelated. Nevertheless, we shall see soon that there is a very close re- 
lationship between these results. To make statements like this, we study abstract 
concepts behind locality theorems. This approach can be seen as the study of 
the essential ideas behind the proofs of locality theorems, rather than using the 
statements of the theorems for proving expressivity bounds. From these theo- 
rems, we extract abstract notions of locality, and show how they are related. We 
also discuss a new form of locality, the bounded degree property, and show how 
it is related to other forms. This property turns out to be particularly simple to 
use in proving expressivity bounds. 
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5.1 Gaifman’s Locality 

We start by analyzing Gaifman’s theorem. This theorem says that only local 
neighborhoods are important for elementary equivalence in first-order logic. This 
is captured by the following definition. 

Definition 1 ( 11321241 1 . A formula ,Xm) is Gaifman-local if there 

exists r > 0 such that for every A G STRUGT [cr] and for every two m-ary 
vectors a,b G , 

tpt^{a) = tp1^{b) implies A |= if(a) if and only if A\= ip{b). 

The minimum r for which this holds is called the locality rank of ip and is denoted 
by \r{ip). 

This definition formulates that if a logic is Gaifman-local, i.e., every formula 
of a logic is Gaifman-local, then indeed only small parts of the input are relevant 
for elementary equivalence in the logic. The part of Gaifman’s theorem that deals 
with sentences, gives rise to the following notion. 

Definition 2 ( [32124] 1. 

— A formula ip{xi, . . . ,Xm), m > 1, is strongly Gaifman-local if there exists 
r > 0 such that for every A^B G STRUGT[cr] and for every two m-ary 
vectors a G A^ and b G B'^ , tp-^{a) = tp^{b) implies A ^ ip{a) if and only 
ifB^iPib). 

— A sentence T is strongly Gaifman-local if it is equivalent to a Boolean com- 
bination of sentences of the form 3yip(y), where ip{y) is a strongly Gaifman- 
local formula. 

Before going further we notice that not every first-order formula is strongly 
Gaifman local. Gonsider the class of directed graphs. Let ip{x) be the formula 
yy-'E{y,x) A 3zyy~'E{z,y). Then ip{x) says that x has not a predecessor and 
there is a point which has no a successor. Suppose that ip is strongly Gaifman- 
local with locality rank r. Let Gi be a chain graph of length r -|- 1 and let G 2 be 
a chain graph of the same length with a loop on the end node. Denote the start 
node of Gi by a^. Then tp^^{ai) = tpf^{a 2 ) but obviously Gi \= ip{a\) whereas 
G 2 Ip{a 2 ). 

From Gaifman’s theorem, we can derive the following. 

Corollary 5 f [1 8j.32|24j L Every first-order formula is Gaifman-local and every 
first-order sentence is strongly Gaifman-local. Moreover, for every ip{x), \r{ip) < 
_ X)/2. □ 



5.2 Hanf’s Locality 

When we studied Hanf’s condition and proofs of Theorems O El and [T] we noticed 
that the essential part in all the proofs was the concept of d-equi valence. We 
now consider this concept in more detail and define the notion of Hanf’s locality. 
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Before giving the definition of the abstract notion we need more notation and 
definitions. 

We extend the notion of d-equivalence (see Section ID for structures with 
parameters. Let A,B € STRUCT[cr] and a G A™, b G B™. Then {A, a) and 
{B. b) are d-equivalent, (A, a) {B, b), if for every isomorphism type r, 

nd{{A,a),T) = nd{{B,b),T). 

In other words, there are as many points a £ A and b £ B such that tp-^{aa) = 
tp^{bb); or equivalently, there is a bijection f : A ^ B such that tp-^(ax) = 
tp^{bf{x)) for all x £ A. 

Definition 3 (| |32I24| L A formula ip{xi, . . . ,Xm) is Hanf-local if there exists 
a number d such that for every A,B £ STRUCT[cr] and for every two m-ary 
vectors a and b of elements of A and B respectively, 

{A, a) ^d {B, b) implies A ^ 'f’io,) if and only if B \= ifib). 

The minimum d for which this holds is called the Hanf locality rank of ip, and 
is denoted by hlr('0). 

Thus, a sentence (p is Hanf-local, if there exists a number d such that A ^d B 
implies A \= p B \= p. From results of the previous section, the following 
theorem is immediate. 

Theorem 8 ( [I32>24J 1. Every sentence p o/ £oou;(Q„)“, or FO -b COUNT, or 
FO is Hanf-local, and hlr(</3) < □ 

This result confirms that although these logics have substantial counting 
power, they can only recognize properties of small neighborhoods, and cannot 
grasp a structure as a whole. The definitions above extract the essential notions 
that were crucial for proving theorems in the previous section. Now these notions 
allow us to compare different locality results. 

5.3 Relationship between the Notions of Locality 

The result below is the main technical lemma that establishes the relationship 
between strong Gaifman-locality, Gaifman-locality and Hanf-locality. It states 
that d-equivalence of structures extends to d-equivalence of structures with pa- 
rameters, if large enough neighborhoods of the parameters are isomorphic. 

Lemma 1 ( | l24j ) . If A ^d B and = tp^j^_^_^(b) , then {A, a) ^d {B,b). 

□ 

Using this, we prove the following. 

Theorem 9 f |.32t24| L Every Hanf-local formula is Gaif man-local. 

Proof. Suppose ip{xi, . . . ,Xm) is a Hanf-local formula with hlr('0) = d. We 
show that Ip is Gaifman-local. Take any two m-vectors a and 6 of a structure 
A such that tpr{a) = tpr{b), where r = 3d -\- 1. Since A ^d A, by LemmaDwe 
obtain (A, a) ^d (A,b). Thus, A ^ ip(a,) if and only A A \= ip(b). Hence ip is 
Gaifman-local and \r{ip) < 3d -I- 1. □ 
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We now consider the relationship between Hanf’s locality and strong Gaif- 
man’s locality. As a technical tool, we need to extend the notion of d-equivalence 
to tuples. The number of different m-tuples whose d-neighborhoods realize an 
isomorphism type r"* of a CTm-structure A, is denoted by nd{A,T^). We write 
A ^m,d B, if for every isomorphism type r"*, 

Equivalently, A ^m,d B if and only if there is a bijection / : A™ — >■ S'" such 
that tp-f{a) = tp^{f{a)) for every a £ A"*. By considering m-tuples whose 
components are the same, we see that for all m > 0, A ^m,d B implies A ^d B. 

Our crucial lemma is that r-equivalence of (m + l)-tuples can be guaranteed 
by d-equivalence of tuples for large enough d that depends only on r. This can 
be shown by considering parametrized versions of these structures and applying 
Lemma [H 

Proposition 1 ( 1321241 ). Let m > 0 and d > 0. Then A ^m, 3 d+i B implies 
A ^m+i,d B. In particular, for every r > 0 and m > 1 there is d such that 
A ^d B implies A ^m,r B. □ 

This can be used to show the following. 

Theorem 10 ( |32II24| L Every strongly Gaifman-local sentence is Hanf-local. 

Consider a sentence T which is equivalent to 3x\ . . . Bxm'f’ixi , . . . , Xm), where 
tp{x) is strongly Gaifman-local. Let r > 0 witness strong Gaifman’s locality of 
-Ip. Take d given by Proposition |T] Then hlr(iJ') < d. Indeed, let A B and 
A \= Then A \= ip{a) for some a £ A™. By Proposition [T] we know that 
A ^m,r B, and thus we find b £ such that tpH^{a) = tp^{b). Since ip is 
strongly Gaifman-local, B |= ip{b) and thus B \= E. Hence, hlr(<f") < d. □ 

This implies that the two parts of Gaifman’s theorem (those dealing with 
sentences and open formulas) are not independent. In fact, for any logic satisfy- 
ing some regularity properties, strong Gaifman-locality of its sentences implies 
Gaifman-locality of its open formulae. See for details. 

5.4 Bounded Degree Property 

One of the easiest ways to prove expressivity bounds is the bounded degree pro- 
perty. It was first introduced for graph queries in studying limits of expressive 
power of database query languages [36j . Later it was generalized to arbitrary 
(finite) structures in [H]. We now review this concept, show its usefulness in 
proving expressivity bounds, and relate it to other notions of locality. 

For a relation Ri in A, we define degree^ (i?^, a) to be the number of tuples 
in Ri whose jth component is a. For directed graphs, this gives us the familiar 
notions of in- and out-degree. The set 



{degree^ (i?i, a) | £ cr, a £ A, j < pj 
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of all degrees realized in A is denoted by deg_set(^). We use deg_count(^) for 
card(deg^et(^)). The class of a-structures A for which deg_set(^) C {0, . . . , fc} 
is denoted by STRUCTfe[CT]. 

Informally, a query has the bounded degree property if an upper bound on 
the degrees in an input structure implies an upper bound on the number of 
degrees realized in the output structure produced by the query. Recall that the 
output of 4’{x\, . . . ,Xm) on A, '0[”4], is the structure with one m-ary relation 
{A, {a G A^ \A\= ^p{a)}). 

Definition 4 ( [lll| i. A formula 'tp{xi,... ,Xm) has the bounded degree pro- 
perty (BDP), if there is a function /^ : N — >■ N such that 

deg -Count {ip[A\) < /^(fc) for any A G STRUCTfe[cr]. 

The bounded degree property is a very useful tool in proving inexpressibi- 
lity results of recursive properties, i.e., for those queries that require fixpoint 
computation. As a simple example, we show that the transitive closure query 
violates the BDP. Assume that TRCL does have the BDP; that is, there is a 
function / : N — >■ N such that deg-CO\mt{TRCL{A)) < f{k) if all in- and out- 
degrees in A do not exceed k. Let N = f{l) + l. Consider A which is a successor 
relation on N points (see Figure |T). Since all in- and out-degrees in A are at 
most 1, we get deg_count( T’i?CL(A)) < /(I) < N, but one can easily see that 
deg_count( TRCL(A)) = N. This contradiction shows that TRCL does not have 
the bounded degree property. This proof also shows that deterministic transi- 
tive closure violates the BDP. (Deterministic transitive closure is defined just as 
transitive closure, except that one only considers paths where each node other 
than the last one has outdegree 1, see [29|.l 

What makes the BDP particularly interesting, is the following result. 

Theorem 11 ( [11] L Every Gaifman-local formula has the bounded degree pro- 
perty. □ 

From results in the previous subsection, we conclude that first-order logic and 
various counting logics we considered have the bounded degree property. This 
confirms out intuition that these logics lack mechanisms for expressing recursive 
(fixpoint) computation. 

Corollary 6 imM)- Every Hanf-local formula has the bounded degree pro- 
perty. In particular, Cooui{Qu)‘^ formulae, FO -|- COUNT formulae (without free 
numerical variables) , and FO formulae have the bounded degree property. □ 

Since deterministic transitive closure does not have the BDP, we obtain the 
following result. 



Corollary 7 ( j32j24j l. Deterministic transitive closure is not definable in 
CooUQuT nor in FO -b COUNT. □ 
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This follows immediately from the BDP, and avoids all the combinatorial 
arguments in Section |4l and especially the ones in Section [3l that are based on 
game-theoretic techniques. 

We now give another example that shows how the BDP can be applied to 
prove inexpressibility results. 

Example J^. A balanced binary tree is a (directed) binary tree in which all paths 
from the root to the leaves are of the same length. Can this property be tested 
in FO, or perhaps in more expressive logics such as /loow(Qu)“? We now use the 
BDP to give the negative answer. 

Suppose that we have a sentence ^ that tests if a given graph is balanced 
binary tree. We next define a query (p{x, y) as follows. It first defines a new graph, 
by interchanging the immediate successors of x, x' and x" , and the immediate 
successors of y, y' and y”, as shown in Figure |3] below, and then it tests is the 
resulting graph is a balanced binary tree. If either x or y fails to have exactly 
two immediate successors, then ip(x,y) will evaluate to false. Assuming ^ is in 
the logic, so is (p{x,y), for logics like FO and £ooi.j(Q„)‘^- 




Fig. 3. Changing successors of nodes in a balanced binary tree 



We now show that (p{x, y) violates the BDP. Assume it does have the BDP, 
and let N = f^{2) + 1. Let A be a balanced binary tree where each path 
from the root to a leaf has length N. Since degrees in A do not exceed 2, 
deg_count((/?[.4]) < by the BDP. We can see that A ^ (p{a,b), for two nodes 
a, 6, if and only if a and b are at the same level in A. Thus, y:>[A\ is a disjoint 
union of N cliques of different sizes, and hence deg_count(</5[.A]) = N. This cont- 
radiction shows that does not have the BDP, and hence cannot be defined in 
>Coouj(Qu)‘^- Consequently, testing for balanced binary trees is not /looa;(Qu)“- 
definable. 

One may have noticed that there is a certain asymmetry in the definition of 
the BDP. In the assumption, we deal with deg_set(A), but the conclusion puts a 
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bound on deg_count((/?[„4]). Can the definition be made symmetric? To formalize 
this, define the strong bounded degree property of as follows: there exists a 
function : N — >■ N such that deg_count(:/5[^]) < /,^(deg_count(^)). 

Proposition 2 (H)- There are first-order definable graph queries that violate 
the strong bounded degree property. □ 

In fact, m shows that even a weaker property is violated by some first- 
order queries. Define the interval bounded degree property of a query ip{x) as 
the existence of a function f^ ■. N ^ N such that deg_count(:^[./l]) < f^{k) 
whenever deg_set(^) C {n, n -|- 1, . . . ,n -\- k} for some number n. Then there 
exist first-order definable queries on graphs that violate this property. 

To summarize, we have seen four different locality conditions: strong Gaifman- 
locality, Hanf-locality, Gaifman-locality, and the bounded degree property. The 
relationship between them is shown in Figure [STl While Hanf-locality is closely 
tied to a game and relatively easy to show for FO and some of its extensions, 
Gaifman-locality and the bounded degree property are very easy to use in ex- 
pressibility proofs. Fortunately, they are implied by Hanf-locality of a logic. 



Strongly Gaifman-local ^ Hanf-local Gaifman-local => BDP 



Fig. 4. The relationship between the notions of locality 



6 Applications in Complexity Theory 

Fagin’s theorem, that equates existential second-order logic and complexity class 
NP, started a new line of research in complexity theory. In the past 20 years, 
many complexity classes have been characterized in logical terms, see [1 3128] for 
an overview. For example, polynomial time and space can be characterized by 
least- and partial fixpoint logics, respectively. Essential for many characterizati- 
ons is the presence of a linear order on the input. The intuition behind having an 
order is simulating the order in which elements of the input appear on the tape 
of a Turing machine. While for order-invariant queries, the exact order does not 
affect the output, its presence is required for a logic to simulate the computation 
of a machine. In fact, it remains an open problem whether there is a logic for 
polynomial time properties of unordered graphs, for example. 

In this section, we deal with a circuit complexity class TG°. This class is de- 
fined via Boolean circuits. Gonsider a family of circuits C = {ci, C 2 , . . . , c„, . . . }, 
where the circuit c„ has n inputs and one output. Given a Boolean string a;, we 
say that C accepts x if the output of on x is 1, whenever x is of length n. 

The class AG° is defined as the class of languages accepted by circuits C 
where each gate is either an AND, or an OR, or a NOT gate, with AND and 
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OR gates having unbounded fan-in (no restriction on the number of inputs) . The 
number of gates in c„ is polynomial in n, and the depth of circuits c„ is constant. 
(More generally, for AC^, the depth of c„ is allowed to be O(log^n).) The class 
TC° is defined as AC°, except that majority gates MAJ are also allowed. Assume 
such a gate has k inputs. Then its single output is 1 iff at least [k\ -|- 1 of its 
inputs are 1. 

The class TC° is not an idle creation of complexity theory; in fact, it is of 
special importance in computer science. It characterizes the complexity of such 
important operations as integer multiplication, division, and sorting, and serves 
as a computational model for neural nets [H]. We refer the reader to survey |1] 
for additional information on circuit complexity. 

Despite its importance, not much is known about the relationship between 
TC° and other complexity classes. We do know that AC° C TC'^, as the parity 
language (strings with even number of Is) is in TC'^, but not in AC° |S]. We also 
know (see jl], for example) that 

TC° C NC^ C DLOG C NLOG C NG C PTIME C NP, 

but we do not know if any of the inclusions is proper! In fact, |45| showed that 
there is inherent difficulty in separating TG° from NP, at least using conventional 
techniques of circuit complexity. A general notion of natural proof was formulated 
in 135 ]; this notion subsumes most of the existing lower bound proofs. Then m 
showed that the existence of a natural proof separating these two classes would 
imply that no good pseudo-random number generators are computable in TG°. 
Putting it in the language of [1] , it would imply that no cryptographically-secure 
functions can be computed in TG*^ , even though cryptographers believe that such 
functions do exist. As the notion of natural proof is quite different from results on 
logical expressibility in finite model theory, one might attempt to avoid obstacles 
of pS] by using a logical characterization of TG°. Below, we survey results in 
this direction. 

Notice that in the definition of classes AG° and TG°, we did not say anything 
about the relationship between circuits c„ G C when n varies; in fact, they can 
compute completely “different” things for different n. However, in most appli- 
cations, those circuits compute the same property, like parity. Gapturing this 
intuition leads to the notion of uniformity. The weakest notion of uniformity is 
PTIME-uniformity, meaning that the mapping n i— >■ c„ is computable in polyno- 
mial time. Similarly, one can define logspace-uniformity (see for using these 
notions with the class TG*^). However, the most widely used notion of uniformity 
is DLOGTIME-uniformity. We spare the reader the more technical definition, 
that can be found in |^, and instead give the characterization theorem. 

Theorem 12 ([5l). DLOGT/ME-MTO/orm TG° = FO -k GOUNT-k < . □ 

From now on, when we speak of TG°, we mean its DLOGTIME-uniform 
version; that is, FO + GOUNT-k <. The latter is the class of problems definable 
by FO-kGOUNT formulae in the presence of an order relation <. We will mostly 
deal with order-independent properties. The notion of order-independence is 
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defined as follows. Suppose we have a FO + COUNT formula ip in the language 
of a and <, and suppose ^ is a cr-structure. Then, for any two orderings <i and 
<2 on A, and for any a, ^ (^(a) iff A<^ ^ where „4< denotes the 

extension of A with the order <. 

Even though we restrict our attention to order-independent properties, the 
mere presence of an order relation does increase the expressive power: 

Proposition 3 ([S])- There are order-independent properties definable inFO-f 
COUNT-f < but not in FO -b COUNT. □ 

The example of a separating query (not the proof!) is quite simple. Let cr = 
(E,U), where E is binary and U is unary. Consider the following property: If 
the interpretation of E is an equivalence relation, then the number of distinct 
sizes of equivalence classes of E equals the cardinality of U. This query is not 
definable in FO -b COUNT [^, but, as shown in [21], can easily be defined with 
order, since all elements whose equivalence classes have the same size, can be 
canonically represented by the <-minimal such element. Then one just checks if 
the number of those elements equals the cardinality of U . 

Theorem reduces the problem of separating TC° from classes above it 
to the problem of logical expressibility; for example, to show TC° yf NLOG, 
it would suffice to show that transitive closure, an NLOG-complete problem, is 
not definable in FO -b COUNT-b <. Since locality gives us an easy proof that 
transitive closure is not in FO -b COUNT, one might try to push the ideas of 
locality into the ordered setting. 

We do not know whether the above expressivity bound on FO-b COUNT-b < 
is true, although we conjecture that it is. Below, we survey some of the partial 
results confirming the intuition. We state the results for the NLOG-complete 
problem of computing the transitive closure, but they also hold for deterministic 
transitive closure, which is complete for DLOG. 

Assume that instead of an order relation, we have a SMCcessor relation SUCC. 
Since it realizes only degrees 0 and 1, as an immediate consequence of the bo- 
unded degree property of FO -b COUNT, we obtain 

Corollary 8 (in)- Transitive closure is not definable in FO -b COUNT -b 
SUCC. □ 

Note that FO plus transitive closure TRCL plus successor relation capture 
NLOG (cf. [T81281 1: hence, FO -b COUNT -b SUCC C NLOG = FO -b TRCL + 
SUCC. This result was first shown in [ 2 ]) via a rather complex argument based 
on games of fSO]. Later, using the results of [40], the journal version of [ 2 ] (see 
121 ) gave a much simpler proof based on Hanf’s condition. Finally, using the 
bounded degree property, we gave a completely elementary proof. 

The use of bounded degree property allows us to substitute any auxiliary 
relation for SUCC, as long as its degrees are bounded by a constant. For example, 
we could use balanced binary trees (note that using such a structure would most 
certainly make a game-based proof unmanageable). The next question is: How 
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can we lift the results for FO + COUNT from the constant world to that where 
degrees are allowed to depend on the size of a structure? 

First such result was given in |32| . and it used the notion of moderate degree of 
m- Let C be a class of structures. Let maxdeg(^{n) denote the maximal degree 
of a structure in C, whose cardinality is n. Then we say that C is a class of 
relations of moderate degree if maxdeg(^(n) < log°^^^ n. That is, for some function 
i5 : N — > N with lim„_>oo = 0, we have maxdeg(^{n) < log"^^"^ n. Combination 
of results from m and led to the following. 

Proposition 4 ( [32] '). Transitive closure is not definable in FO + COUNT in 
the presence of relations of moderate degree. □ 

A linear order on an n-element set realizes n different degrees, from 0 to n— 1. 
Thus, we need to lift the results from relations of small (constant or moderate) 
degree to relations of large (comparable with the size of the input) degree. The 
concept of moderate degree was introduced in HZ] to show that connectivity 
is not definable in monadic E\ in the presence of those relations. Later, m 
extended this to linear orders. Thus, one may ask if a similar avenue of attack 
on the separation problem can be pursued in the case of FO + COUNT. 

A partial result in this direction exists. Let Ok stand for the class of relations 
which are pre-orders (A, -<) (i.e., ^ is reflexive and transitive), and each equiva- 
lence class of the relation x ^ y = {x < y) A {y < x) has at most k elements. In 
particular, Oi is the class of linear orders. 

Theorem 13 ( [38] L Transitive closure is not definable in FO-l-COUNT-l-Ofe, 
for any k > 1. □ 

In fact, the result of is even stronger as it allows relations from O 2 to be of 
special form: those that have at most g{n) elements in equivalence classes of size 
2, and the rest, n — g(n) elements, form a linear order. The function g(n) can be 
chosen to be arbitrarily small, e.g., log log . . .logn, but it cannot be bounded by 
a constant. The version of Theorem 1131 proved in 1381 for deterministic transitive 
closure, implies DLOG ^ FO -I- COUNT -|- Ok for any fc > 1. But this still falls 
short of resolving the most important case of fc = 1. 

One of the main goals of pursuing this line of research was to avoid the 
obstacles on the path towards separating TC° from other classes posed by the 
natural proofs of m- At the first glance, our expressivity bounds look nothing 
like natural proofs, although we must admit that there was no systematic study 
conducted on the relationship between classical lower bounds proofs in circuit 
complexity and logical expressivity bounds for FO -I- COUNT and the likes. 

However, while avoiding one kind of problems, we encountered (perhaps even 
created) different ones. For example, the proof of Theorem [T3] is based on the 
following technique. One first shows that if for a class C of auxiliary relations, 
there exists a class of graphs Q such that the pair (C, Q) satisfies two properties, 
PI and P2, then transitive closure of graphs in Q is not definable in FO -I- 
COUNT +C. Then, [3S] showed how to construct Q for C being Ok, with fc > 
1, thus proving inexpressibility of transitive closure in FO -I- COUNT + Ok- 
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Unfortunately, [38j also showed that if C = Oi, the class of linear orders, then 
there is no class of graphs Q such that (C, Q) satisfies PI and P2. That is, the 
approach is inherently limited for showing results in the ordered setting. 

Another large obstacle to using the ideals of locality for proving the separa- 
tion is a recent result: 

Theorem 14 (|24p. There exist order-invariant formulae in FO-I-COUNT-I- < 
that do not have the bounded degree property. Consequently, order-invariant FO-I- 
COUNT-b < is neither Gaifman-local nor Hanf-local. □ 

Nevertheless, we believe it was a useful exercise to study the expressivity of 
FO -b COUNT with relations of large degree, viewing it as a reasonable approxi- 
mation of uniform TC*^. The bounds of HD, obtained less than 4 years ago with 
a great deal of effort, were shown here by a simple application of the bounded 
degree property. The best lower bound given by locality arguments (Theorem 
HU shows that one can get very close to a linear order, and thus very close to 
uniform TC°, without achieving the power of logspace computation. In a way, 
results of this section can be viewed as separating “very uniform” versions of 
TC° from DLOG and classes above it. 

7 Applications in Database Theory 

The theory of database query languages is firmly grounded in finite-model theory, 
and also provides a major motivation for finite-model theory research. Traditio- 
nal databases query languages, such as relational algebra and calculus, have 
precisely the power of first-order logic. An important subclass, called conjunc- 
tive queries, is simply the {3, A}-fragment of first-order logic. Various extensions 
of relational calculus, such as Datalog, Datalog with negation, and the language 
of while loops, correspond to various fixpoint extensions of first-order logic, see 

PP- 

In the relational model, data is stored in relations. For example, a database 
may have two relations, for storing information about employees (called Emp) and 
departments (called Dept). Assume that Emp stores triples containing employee 
name, department, and salary, and Dept stores triples containing department 
name, manager, and manager’s salary. Below is an example of a database: 



EName 


EDept 


ESalary 


John 


A1 


50 


Ann 


A1 


60 


Jim 


B2 


75 



DName 


Manager 


MSalary 


A1 


Bob 


80 


B2 


Steve 


85 


C3 


Mary 


80 



An example of a query is “For each employee, find his or her manager. ” This 
can be written in first-order logic as: 

q{emp, manager) = 

3dept, esal, msal Emp(emp, dept, esal) A Dept(depf, manager, msal). 
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In real life, of course, programmers do not write first-order formulae; instead 
they write queries in the language called SQL, which is the lingua franca of the 
commercial database world. The above query, in SQL, will look like: 

SELECT E.EName, D.McOiager 
FROM Emp E, Dept D 
WHERE E . Edept = D . Dname 

The best way to read such statements is as set-theoretic comprehensions: the 
above becomes 



{EName, D. Manager) 



{EName,EDept,ESalary) € Emp, 

{D Name, Manager, MS alary) € Dept, 
EDept = DName 



The basis of SQL is the select-from-where statement, with the addition union 
and difference, and features such as view creation, which make the language com- 
positional. This basis has precisely the power of the first-order logic. However, all 
practical implementations of SQL come equipped with two additional features: 
arithmetic operations, and aggregate functions. For example, consider the follo- 
wing query: “Eind all departments that have more than 5 employees, together 
with the name of the manager and the average salary of the employees. ” 

In SQL, this will be written as 

SELECT D.DName, D. Manager, AVG(E.ESalary) 

FROM Emp E, Dept D 

WHERE E . Edept = D . Dname There are two key new 

GROUPBY D . DName 

HAVING COUNT (E.EName) >5 

features in this query. Grouping is given by the clauses GROUPBY and HAVING: 
for each department manager, we group together all the employees in his/her 
department, provided there are more than five of them. The other feature is 
aggregate function: these are AVG, for computing the average salary, and COUNT, 
for counting the number of employees. 

Let us now see why adding these features is indeed an extension of first-order 
logic. We consider a query that we know is inexpressible in first-order logic alone: 
given a graph G, find the set of nodes x with in-deg(x) = out-deg(x). We let 
graphs be represented as a binary relation Edges with two attributes. From and 
To. As the first step, we create two new relations, one storing nodes together 
with their in-degrees, and the other storing nodes with their out-degrees. Such 
intermediate relations are called views in SQL, and are created as 



CREATE VIEW INDEGV(Node, Indeg) AS 
SELECT E.To, COUNT (E . From) 

FROM Edges E 
GROUPBY E.To 
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CREATE VIEW DUTDEGV(Node , Outdeg) AS 
SELECT E.From, COUNT (E. To) 

FROM Edges E 
GROUPBY E.From 

Using these views, one computes the answer to the query as 

SELECT INDEGV.Node 

FROM INDEGV , OUTDEGV 

WHERE (INDEGV.Node = OUTDEGV . Node) 

AND (INDEGV. Indeg = OUTDEGV. Outdeg) 

By now, the reader must be convince that Hartig and Rescher quantifiers 
can be expressed in SQL. Thus, it is more powerful than first-order logic. The 
question now is: 

How expressive is SQL? 

More precisely, there seems to be a “folk result” saying that SQL cannot com- 
pute recursive queries, such as transitive closure. That is, it lacks a mechanism 
for recursive computation. The question now is: What kind of formal statement 
can one prove to confirm this intuition? 

The approach of some textbooks is to restrict SQL to its subset which is 
essentially first-order logic, and use expressivity bounds for the latter. However, 
as we have just seen, this is not satisfactory. The difficulty with answering the 
question above is that there are dozens of different version of SQL (see [l^ for 
an overview of standards and dialects), and they often support different sets of 
operators. For example, some versions even add the transitive closure operator. 
Thus, we restrict our attention to features that are common to all versions 
of SQL; that is, grouping and aggregation (cf. |1I43| ). The first result on the 
expressive power of such a language was based on yet unproven assumption 
from complexity theory, and two observations: SQL queries can be evaluated in 
deterministic logspace, and transitive closure is complete for nondeterministic 
logspace. Thus, 

Proposition 5 ( |10J ). Assume that DLOGSPACE ^ NLOGSPACE. Then tran- 
sitive closure cannot be expressed in SQL. □ 

Can we get rid of the unproven assumption? The problem we face is that SQL 
per se is quite inconvenient to work with - its syntax is quite awkward, and in 
fact it has been an object of persistent criticism. SQL combines sets and multisets 
in order to evaluate aggregates: for example, computing the average salary, one 
cannot first project out the salary attribute and then compute its average, as 
the elimination of duplicates will produce an incorrect result. Following EOEg, 
a language that deals correctly with multiset and set semantics was proposed in 
|3B[. As this language can model the main features of SQL, and extends first- 
order logic, it was suggested to use it as a rational reconstruction of SQL. 
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Let us now give an informal introduction into this language, which we call 
here AGGRq. It deals with objects which can freely combine rational numbers, 
elements of the domain of atomic values, D, tuples, and sets. In particular, it 
permits sets of sets. The language is statically typed (cf. I2T1MI ), but we shall not 
go into detail here. The full description can be found in [2Tt36l : here we highlight 
the salient features. The language allows one to apply a function to each element 
of a set, that is, obtain a set {f{x) \ x G X} from X. If all elements of a set X are 
sets themselves, their union can be taken: ^ ■ Basic arithmetic operations 

(+, — , *, -G, <) are available on Q. On the domain of basic values D, only equality 
test is available (notice the absence of order). Finally, if / is a function into Q, 
its values on a set X can be added up; that is, one can compute fix)- 

Note that if / is identically 1, then the above is the cardinality of X. 

It was proved in m that these features model the main features of SQL. 
Furthermore, extending a result from m, a conservativity property was shown 
in m- It says that nesting of sets is in a sense superfluous: every query from 
relational databases to relational databases in AGGRq can be written in a way 
that does not use sets of sets. Nesting is essential for modeling grouping, and 
thus the (nontrivial) conservativity result gives us a rather pleasant language 
(without higher-order features) to model all the features of SQL. In [^, the 
following was proved: 

Proposition 6 ([36]). Transitive closure cannot be expressed in AGGRq. □ 

While this does provide useful bounds, there are two problems with the proof 
in m- First, it is very syntactic. It proceeds by establishing a normal form result 
for queries on a special class of inputs; the property depends both on the class of 
inputs, and the properties of the chosen syntax of the language. Thus, making 
a minor change in the syntax that does not affect expressiveness would mean 
that the proof must be redone from scratch. Even more unpleasantly, the proof 
in m establishes inexpressibility of transitive closure, but fails to establish a 
general property that will give us expressivity bounds. 

In [21136] it was conjectured that relational queries in AGGRq have the BDP. 
By relational queries we mean those whose inputs and outputs only contain 
elements of D, but no numbers, although rational numbers can be used in the 
process of evaluating a query. An SQL example of finding nodes with equal 
in- and out-degrees shows that there are relational queries definable in SQL 
but not in first-order. Clearly, proving the above conjecture would resolve the 
problem for queries such as transitive closure or deterministic transitive closure. 
By extending the normal form result AGGRq, the following was proved. 

Proposition 7 ( [11 J L Every relational query in AGGRq has the bounded degree 
property. □ 

It is still unpleasant that the proof depends on a particular syntax for the 
language. Also, it would be interesting to know if relational queries are Gaifman- 
local. These two questions were considered in [32j . The approach taken by m 
was the following. Restrict the language to natural numbers only. That is, one 
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can compute COUNT but not AVG. Let us call this language AGGR^. With rationale 
out of the way, try to embed it into FO + COUNT to prove locality. 

Embedding into FO + COUNT turns out to be problematic, as in FO + 
COUNT the sizes of the first-sort and second-sort universes are the same. In 
contrast, in SQL, one can create numbers much bigger than the size of the 
database. A simple example is this: consider an n-element set X, and compute 
'l^xex /(*) where f{x) = 2 for every x. The result is 2n. The solution to the 
problem is to modify a query, essentially by putting a huge linear order “on the 
side” and having all arithmetic done on that linear order. This technique led to 
the following result. 

Proposition 8 ( [32J L For any relational query Q in AGGR^, another query 
Q' can he found such that (a) Q is Gaifman-local iff Q' is, and (b) Q' can 
be defined in FO -I- COUNT. Consequently, every relational query in AGGRn is 
Gaifman-local. □ 

Using this technique, m returned to the main question: what is the ex- 
pressive power of SQL? It defined a new language, called AGGR^®"^, which first 
restricts AGGR^ to objects that do not contain sets of sets, and then adds new 
arithmetic operation, and product over a set. That is, given a set X and a func- 
tion / into N definable in AGGR®^^, one can compute /(^) AGGR®^^. 

Then the following sequence of results was proved. 

Theorem 15 f [37p. 1) Every relational AGGRq query can he expressed in the 
language AGGR^®'^. 

2) For any relational query Q in AGGR®^^, another query Q' in AGGR®®"^ can 
be found such that (a) Q is Gaifman-local iff Q' is, and (b) Q' can be defined in 
FO-b COUNT. □ 

The bulk of the proof of this theorem is in showing (b) of part 2) . Combining 
these results with locality of FO -I- COUNT, we obtain: 

Corollary 9. Every relational query in AGGRq is Gaifman-local. □ 

Consequently, plain SQL queries are Gaifman-local and have the bounded 
degree property. Therefore, queries that need a recursion mechanism (transi- 
tive closure, deterministic transitive closure, connectivity test, etc.) cannot be 
computed in SQL. 

Thus, locality helped us answer important questions about expressive power 
of real world database query languages. 

We conclude this section by a remark about the set of basic operations on 
the domain of atomic values, D. In all the above results, we assume that it 
is only possible to test if two elements of D are equal. In many applications, 
there is a meaningful linear order on such domains (for examples, salaries can be 
compared). What happens if D is ordered? As it often happens, things become 
a lot more complicated in the ordered setting. In a way, we saw it in the last 
section. For SQL, one can show that it is possible to express any query from 
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FO + COUNT+ < if Z? is ordered. Thus, every uniform TC° property becomes 
definable in SQL, and hence the problems of expressivity in of recursive queries 
(such as transitive closure) hinges on the separation of complexity classes, thus 
confirming the original intuition of |10| . Note that there are other well-known 
examples of close connection between separation of complexity classes and ex- 
pressivity bounds of query languages, see [2]. 

One can go further and add more operations to D. For example, one can 
assume that D is the field of real numbers with the usual operations -1-, , or 

perhaps more complex such as e^. The results of |7] show that for the class of 
generic [1] queries (those that commute with permutations of D), these extra 
operations do not add expressive power, beyond a linear order. Most examples 
of queries we consider - transitive closure, connectivity test, etc. - are generic. 
Thus, adding extra operations beyond < does not lead to an increase in power 
of languages such as first-order logic or SQL. This assertion, in the case of an 
interpreted domain D, depends on what one means by quantification 3a;. In most 
application, quantification in queries assumes the finite database; however, it is 
conceivable (and in some applications, important) to quantify over D. We refer 
the reader to [Zj for the discussion on this topic. 

8 Conclusion 

In this paper, we reviewed some results that were developed for proving lower 
bounds for logical expressibility. We considered first-order logic and its exten- 
sions with several kinds of counting mechanisms. We presented the usual game- 
theoretic characterizations of those logics, as well as Gaifman’s and Hanf ’s theo- 
rems, and general notions behind these results. We also studied the relationship 
between these notions and the bounded degree property. We reviewed applica- 
tions of these notions in descriptive complexity theory and database theory. 

Note. Several results on locality of logics appeared recently. As it is too 
late to include their detailed descriptions in this survey, we just give pointers to 
relevant papers. Most importantly, [1^ partly extended locality techniques to the 
ordered setting. Namely, m proves that every order-invariant query definable in 
FO-f < is Gaifman-local, where the concept of neighborhood does not take into 
account the order relation. An elementary proof of Gaifman-locality of a very 
powerful counting logic was given in ; that paper also used a refined argument 
for proving locality to find tight bounds on the radii of neighborhoods required 
in locality theorems. Here, we typically used the radii of the order 0(3'^’"^'^^); 
jHHj l improves this to 0(2'?’'*^'^^). Hanf- locality of the counting logic from [33] 
was proved in m- It was also shown in that Hanf-locality remains valid 
under the addition of aggregate functions, which provided a transparent proof 
of Gorollary |3] Finally, [34] improved Theorem [T3] by showing that invariant 
queries in FO 3- GOUNT + Ok, k > 1, have the bounded degree property. This 
remains true for more powerful logics, and a more general class of auxiliary 
relations. 
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Abstract. This paper presents a perspective on the relationship bet- 
ween Lindstrom quantifiers in model theory and oracle computations in 
complexity theory. We do not study this relationship here in full gene- 
rality (indeed, there is much more work to do in order to obtain a full 
appreciation) , but instead we examine what amounts to a thread of rese- 
arch in this topic running from the motivating results, concerning logical 
characterizations of nondeterministic polynomial-time, to the conside- 
ration of Lindstrom quantifiers as oracles, and through to the study of 
some naturally arising questions (and subsequent answers). Our presen- 
tation follows the chronological progress of the thread and highlights 
some important techniques and results at the interface between finite 
model theory and computational complexity theory. 



1 Introduction 

Prior to about 1974, the model theory of finite structures had been deemed to 
be rather uninteresting, essentially because many of the fundamental results of 
(infinite) model theory, such as the Completeness and Compactness Theorems, 
fail when only finite structures are allowed (see [17]). This is not to say that 
there are no interesting results in the finite case: Trakhtenbrot’s Theorem |3H] 
and the 0-1 law for first-order logic m are two. But such results appeared spo- 
radically and there was no real concerted research effort in this regard. However, 
all this changed thanks to Fagin’s Theorem [lO] which tied together finite model 
theory and computational complexity theory. Since then, finite model theory has 
witnessed an explosive growth. 

Notwithstanding the beauty of Fagin’s Theorem, finite model theory still 
remained relatively dormant until the late seventies and early eighties. Instru- 
mental in those early days was the work of Immerman which provided further 
links between finite model theory and complexity theory; and, in particular, 
yielded logical characterizations of numerous complexity classes by extensions 
of first-order logic using “operators corresponding to problems” (other opera- 
tors of a very different syntactic nature, such as fixed-point operators, were 
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also used to extend first-order logic [I20l39p . These operators are essentially in- 
finite uniform sequences of Lindstrom quantifiers: Lindstrom | 24I25| introduced 
these quantifiers in model theory so as to characterize minimal logics expressing 
certain properties. Immerman’s logical characterizations of complexity classes 
yielded other things, such as new notions of (complexity-theoretic) reductions 
and strong (complexity-theoretic) completeness results, and his general approach 
enabled him to solve a long-standing open question; namely, he showed that the 
complexity class NL is closed under complementation [221 (this problem was 
solved independently at about the same time by Szelepcsenyi m\)- 

Whilst Immerman logically characterized complexity classes such as L, NL 
and P using operators of the form alluded to above, NP remained to be so 
characterized. Such a characterization was explicitly established by Stewart [31]; 
although unbeknownst to him, it had (essentially) already been obtained by 
Dahlhaus jSj. It is this logical characterization of NP that forms the starting 
point for the research travels in this paper. In more detail, we intend to show 
how this logical characterization of NP leads to the consideration of Lindstrom 
quantifiers as oracletfi] (even though there are very definite differences between 
the two concepts), and to a logical characterization of the complexity class 
We then examine this characterization in more general settings in two ways: 
first, we consider analogous logical characterizations of complexity classes of the 
form where CC is not necessarily NP (that is, we explore the extent to 

which the methods and techniques involved in the logical characterization of 
are in some sense generic); and, second, we consider whether our logical 
characterization of , which happens to be on the class of ordered structures, 
holds on the class of all structures. 

It is not our intention here to provide a systematic and detailed framework 
within which different oracle access mechanisms for oracle machines of varying 
resource bounds can be related to the use of Lindstrom quantifiers in different 
logics: one such framework has been laid out in What we aim to do 

is to show how logical characterizations of NP give rise to the study of the 
relationship between resource-bounded oracle computations and logics involving 
Lindstrom quantifiers. Note the wording of the previous sentence: the motivating 
spark for the research highlighted within this paper is essentially of the form: 
“ We have a logical characterization of NP as a fragment of a particular logic. 
So which natural complexity class, if any, is captured by the full logicT' . That 
is, the research arose because we wanted to find out which complexity class 
was captured by a certain logic, and not because we decided to model resource- 
bounded oracle computations in a logical fashion. 

The survey presented herein highlights results in the papers I7II5I32I33I . 
Whilst we acknowledge that survey papers generally include explicit definitions 
of all concepts and notions occurring, we occasionally fail to be as detailed as 
we might be both for reasons of space and because it is really on the results in 

^ We do not mean to imply that this logical characterization of NP inspired the first 
consideration of Lindstrom quantifiers as oracles: such a consideration was made by 
Gradel |13| . 
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the aforementioned papers that we wish to focus. There are already a number of 
surveys in which any such material missing from this paper might be found (not- 
ably H^), as well as the standard finite model theory reference [^. We assume 
that the reader is acquainted with the basic notions of complexity theory (as 
can be found in, for example, m)- We include sketches of important proofs and 
we hope that we give a flavour of how current research at the interface between 
finite model theory and computational complexity theory is progressing. We also 
pose a number of problems which we believe to be worthy of investigation. 

We begin, in Section 2, by giving some basic definitions relevant to the logi- 
cal characterization of complexity classes before saying more about Lindstrom 
quantifiers and how they compare with oracles in Section 3. We exhibit a logical 
characterization of the complexity class on ordered structures, in Section 

4, and in Section 5, we examine the logical characterization of other log-space 
oracle complexity classes. In Section 6, we return to our logical characterization 
of and consider it on the class of all structures. 

2 Some Basic Definitions 

We begin by giving some very basic definitions relating to how to consider a 
complexity class as a class of problems, where by “problem” we mean a set of 
finite structures (see below) . The reader is encouraged to consult [9| and [TT] for 
further details regarding finite model theory and complexity theory, respectively. 

In general, a signature a = ^^ 2 ) • • ■ ^ tuple of 

relation symbols {R^ : i = 1,2, .. . ,r}, with of arity a^, and constant sym- 
bols {Cj : i = 1,2,... ,c}. A (finite) structure of size n over ct is a tuple 
A = ({0, 1, . . . , n — 1}, i?i, i? 2 , . . . , Rr, Cl, C 2 , . . . , Cc) consisting of a universe 
\A\ = {0,l,...,n— 1}, relations i?i,i? 2 ,--- , Rr on the universe |.4| of arities 
ai, 02 , . . . , ttr, respectively, and constants Ci, C 2 , ... , Cc from the universe |.4|. 
The size of A is also denoted by |^| . We denote the set of all (finite) structu- 
res over a by STRUCT((t) (henceforth, we do not distinguish between relations 
(resp. constants) and relation (resp. constant) symbols, and we assume that all 
structures are finite and of size at least 2). A problem over cr is a subset of 
STRUCT((t) which is closed under isomorphism. If fl is some problem then the 
signature of fl is denoted cr(I2). 

We tie together classes of problems and complexity classes as follows. Comple- 
xity classes are generally considered to be classes of languages over the alphabet 
{0,1}, recognized by various resource-bounded computing devices (for example, 
by log-space deterministic Turing machines), whereas problems are (isomorphism 
closed) sets of finite structures. What we do is to encode, via some (reasonable) 
encoding scheme (see below), our structures so that they might be input to some 
computational device. For example, suppose that we have some structure A of 
size n in which there is a relation R of arity a. We could encode this relation as 
a string of bits 

i?(0,0,... ,0,0),R(0,0,... ,0,1),... ,i?(0,0,... ,0,n-l), 

i?(0, 0, . . . ,1,0),... , R(n — l,n— 1,... ,n — l,n — 1). 
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Any constant of A could be encoded as a 0-1 string representing its binary 
representation, and these strings, encoding relations and constants of A, could 
then be concatenated, perhaps, to obtain an encoding of A. Note that there 
are numerous (an infinite number, in fact) different encoding schemes we might 
choose to encode the structure A. Consequently, we can now say that some 
problem is accepted by, for example, some Turing machine if some encoding of it, 
according to some fixed encoding scheme, is the language accepted by the Turing 
machine. By “fixed” we do not mean that the same encoding scheme should be 
used for every problem, just that the same encoding scheme should be used for 
every structure of our problem. Note also that because a problem is a set of 
structures closed under isomorphism, there will be a number of strings encoding 
the “same” structure. (This is the basic approach adopted in complexity theory 
when abstract decision problems are equated with languages: see HD)- 

We are now in a position to identify a complexity class CC with a class of 
problems L. We identify CC with L, and write CC = L and say that L captures 
CC, if, and only if: 

• for every problem in L, there is an encoding of this problem in CC; and 

• for every language in CC, there is a problem in L of which this language is 
the encoding. 

We reiterate again that any encoding scheme is only specific to the problem in 
hand. 

Given any language A over {0, 1}, we can always obtain a problem f2 of 
which this language is the encoding; and, moreover, this problem is over the sig- 
nature a = (M,S), where M is a unary relation symbol and 5" is a binary 
relation symbol. For any string oj G A of length n, let .4 be a a-structure 
of size n such that S encodes a successor relation on {0,1,... ,n — 1}, e.g., 
{(0, 1), (1, 2), (2, 3), . . . , (n — 2, n— 1)} which yields the ordering 0, 1, 2, . . . , n— 1, 
and such that M{i) holds if, and only if, the ith bit of w, with respect to the 
ordering given by S', is 1. Note that there are numerous such structures A corre- 
sponding to oj in this way. Let the problem L2 consist of every such cr-structure 
derived from some string u> G A in this way (and so 17 is closed under isomor- 
phism) . As our encoding scheme for 17, encode any u-structure A for which S is 
a successor relation as the string obtained by reversing the above process, and 
encode any cr-structure A for which S is not a successor relation as some fixed 
string not in A (we may assume that there exists such a string). Then, via this 
encoding scheme, the problem 17 is encoded as the language A. We shall return 
to these “successor” structures presently. 

The above discussion suffers from the fact that we have not defined exactly 
what we mean by an encoding scheme. Rather than becomes embroiled in this 
issue, let us refer the reader to the discussion in m on as to what constitutes a 
“reasonable” encoding scheme. 

We remark that it is usual to say that an input to some Turing machine 
has size n if the input string consists of n symbols. We always work modulo 
an encoding scheme and consequently we talk of some structure being input to 
some Turing machine when strictly we mean that the encoding of this structure 
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is input. Furthermore, if we say that some structure of size n is input to some 
Turing machine then “size” refers to the size of the structure and not the length 
of the encoding. 



3 Lindstrom Quantifiers in Descriptive Complexity 

We now show how Lindstrom quantifiers are used in the logical characterization 
of complexity classes defined by a model of computation not involving oracles. 
Prompted by these characterizations, we then examine the possibility of using 
oracle machines to solve problems definable in other related logics formed using 
Lindstrom quantifiers. 



3.1 In the Absence of Oracles 

First-order logic, FO, consists of all formulae formed from atomic formulae (over 
some signature) using the boolean connectives A, V and and the quantifiers V 
and 3. We write FO to denote the class of first-order formulae and also the class 
of problems defined by first-order sentences, and do likewise for any other logic. 
Whilst it is not difficult to see that any problem in FO is in the complexity class 
L, there are problems in L that are not in FO; one such being the problem, over 
the empty signature, consisting of those structures of even size. Consequently, 
if we are interested in capturing complexity classes such as L, NL, P and NP 
then first-order logic obviously needs to be extended in some fashion. 

One natural extension is to second-order logic, SO, which is obtained by al- 
lowing the existential and universal quantification of new relation symbols (that 
is, relation symbols not appearing in the underlying signature). For example, 
if (72 = {E), where if is a binary relation symbol (and so (T 2 -structures can be 
thought of as undirected graphs via “there is an edge from vertex u to vertex v 
if, and only if, E{u,v) or E{v,u) holds”; and also as directed graphs via “there 
is an edge from vertex u to vertex v if, and only if, E{u,v) holds”) then the 
problem 

3COL = {A G STRUCT((T 2 ) : the undirected graph with vertex set 
given by |A| and edge set given by the relation E can be 3-coloured} 

is defined by the second-order sentence 

3R3W3B{yx{R{x) V W{x) V B{x)) 

/\'ix{^{R{x) A W{x)) A ~^{R{x) A B{x)) A ~^{W{x) A B{x))) 
MxMy{{E{x,y) V E{y,x)) ^ {~^{R{x) A R{y)) A ^{W{x) A W{y)) 

^^{B{x)^B{y))))) 



(where R, W and B are new relation symbols af arity 1). 
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In a seminal result, Fagin |10J proved that a problem is in NP if, and only 
if, it can be defined by a sentence of existential second-order logic, El (that is, 
the fragment of SO consisting of formulae of the form 



3Ti 3T2 . . -3Tk(l), 



where each Ti is a new relation symbol and is a first-order formula) . Stockmeyer 
m extended Fagin’s characterization to show that SO captures the Polynomial 
Hierarchy, PH. 

Given the facts that FO C L and that El is a rather basic fragment of SO, 
one still has the problem of capturing complexity classes “below” NP such as L, 
NL and P. One method is to restrict the syntax of El even further, as was done 
by Gradel m, but one can also augment FO with appropriate “operators” . We 
illustrate such an approach with the best known of these operators, TG. 

Define the signature (T2++ = (E,C,D), where E is a binary relation symbol 
and C and D are constant symbols, and define the problem TG as 

TG = {A G STRUGT((T 2 -i-+) : the digraph with vertex set |^| and 

edge set given by the relation E contains a path from vertex C 
to vertex £>}. 

Gorresponding to the problem TG is an operator of the same name; that is, an in- 
finite uniform, or vectorized, sequence of Lindstrom quantifiers (whilst we do not 
define here explicitly what a Lindstrom quantifier is, we hope that the essence of 
Lindstrom quantifiers is gleaned from what follows). The logic (±TG)*[FO], or 
transitive closure logic, is the closure of FO under the usual first-order connec- 
tives and quantifiers, and also the operator TG, with TG applied as follows. Given 
a formula ^(x, y) G (±TG)* [FO] , where the variables of the fc-tuples x and y, for 
some k, are all distinct and free in if, the formula <P defined as TG[Axy^](u, v), 
where u and v are A:-tuples of (not necessarily distinct) constant symbols and 
variables, is also a formula of (±TG)*[FO], with the free variables of being 
those variables in u and v, as well as the free variables of 4> different from those 
in the tuples x and y. If is a sentence then it is interpreted in a structure 
A G STRUGT(cr), where a is the underlying signature, as follows. We build a 
digraph with vertex set \A\^ and edge set 

{(a, b) G \A\^ X \A\^ : ^!>(a, b) holds in A}, 

and say that A\^ <P ii, and only if, there is a path in this digraph from vertex u 
to vertex v (the semantics can easily be extended to formulae of (±TG)*[FO]: 
see, for example, m for a more detailed semantic definition). We also denote 
the fragments of (±TG)*[FOj where applications of TG do not appear within 
the scope of a negation sign by TG*[FO], and where at most m applications 
of TG may be nested by (±TG)"*[FO]: the fragment TG^pO] is as expected. 
We reiterate that TG is essentially an infinite sequence of Lindstrom quantifiers 
{TGfc} where TGfc binds 2k free variables in the formula to which it is applied. 
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In a celebrated result, Immerman |21l22j captured the complexity class NL 
by the logic (±TC)*[FO], but only on the “successor” structures, or to give 
them their popular name ordered structures, encountered earlier. More precisely, 
he assumed that the logic (±TC)*[FO] has at its disposal a “built-in” binary 
relation symbol succ, different from any symbol of the underlying signature, that 
is always interpreted as a successor relation (i.e., succ{x,y) holds if, and only if, 
y = x + 1) on the domain of any structure, and also two constant symbols, 0 and 
max, that denote the least and greatest elements with respect to the relation 
succ. we denote the resulting logic by (±TC)*[FOs]. Note that sentences of a 
logic such as (±TC)* [FOs] might not define problems, i.e., sets of finite structures 
closed under isomorphism, as, for example, the successor relation might be used 
“explicitly” in a sentence as it is in the sentence E{0,max) of FOs(tT 2 )- We 
simply ignore all such sentences and only ever concern ourselves with sentences 
defining problems. 

However, it is undecidable as to whether a sentence of (±TC)*[FOs] defines 
a problem and consequently (±TC)*[FOg] does not have a recursive syntax and 
should not really be called a logic (see m)- Notwithstanding this remark, we 
continue to call (±TC)*[FOs] a logic on the grounds that one could regard it as 
the (bona fide) logic (±TC)*[FO] restricted to the class of ordered structures. 
But let us press on with our discussion of Immerman’s characterization of NL. 

In fact, Immerman showed that (±TC)*[FOs] = TC^[FOs] = NL and that 
every formula in (±TC)*[FOs] has a simple normal form; in particular, any 
problem SI that is definable by a sentence of (±TC)*[FOs] can be defined by one 
of the form 



TC [Axy ^/>] (0 , max) , 

where: x and y are fc-tuples of distinct variables, for some k] ip is a, quantifier- free 
formula of FOs (actually, a quantifier-free projection: see |2I]); and 0 (resp. max) 
is the constant symbol 0 (resp. max) repeated k times. In such a circumstance we 
say that there is a quantifier-free first-order translation with successor from SI to 
TC; and as S2 is an arbitrary problem in NL we have that TC is complete for NL 
via quantifier-free first-order translations with successor (clearly, we may have 
other translations such as first-order translations with, or without, successor). 
Such a result sharpens the well-known result that TC is complete for NL via 
log-space reductions [22]. An important point to note is that adopting such a 
logical viewpoint to complexity theory yields techniques for proving problems 
complete for some complexity class that were hitherto unavailable (see [31]) and 
scope for showing that traditional complete problems might not remain complete 
under restricted logical reductions (see [ID- 

One can augment FO (or FO^) with an operator (or operators) such as TC 
corresponding to any problem (or problems) and examine the class of problems 
so captured. A variety of such logics have been formed and many well-known 
complexity classes, including L, NL, P and NP, consequently captured (see, 
for example, the presentation and references in E3)- Of particular interest 
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to us is the logic formed by extending FO^ using an operator corresponding to 
the problem 

HP = {A € STRUCT(ct 2 ++) : the digraph with vertex set given by 
1 .4 1 and edge set given by the relation E has a Hamiltonian 
path from vertex C to vertex D}. 

Theorem 1. | 5I31| A problem is in NP if, and only if, it can be defined by a 
sentence of the logic HP*[FOs]- 

In fact, Theorem [T] holds in the absence of the built-in successor relation 
1^, and there is a normal form result analogous to that for TC*[FOs], above 
1^ . Obviously a more satisfactory result would be that NP = (±HP)*[FOs]. 
However, this would yield as an immediate corollary that NP = co-NP! Con- 
sequently, we are left with the question: “Can we characterize the class of pro- 
blems {±HP)*[FOs\ in the traditional {Turing-machine-based) settingT' . Note 
that such a question does not arise with regard to the analogous logical cha- 
racterizations of L, NL and P as these complexity classes are closed under 
complement ation . 

3.2 Modelling Lindstrdm Quantifiers Using Oracles 

Before turning to the question just stated, let us consider the possibilities of 
modelling applications of Lindstrom quantifiers using oracles. For concreteness, 
let us consider a sentence (j) of the logic (±HP)*[FOs]. There are some points 
worthy of note, which we return to presently. 

(а) There might be a number of applications of HP within (j) and some of these 
applications might be nested. 

(б) When deciding whether some appropriate structure, of size n, satisfies </>, 
every application of HP involves the building of a digraph of size n^, for 
some k. 

(c) In constructing (f>, the first application of the operator HP (in the process of 
constructing (f) is to a first-order formula. 

Let us consider a typical standard oracle-machine computation: for concre- 
teness, consider a log-space deterministic oracle Turing machine (DOTM) where 
the oracle is the problem HP. In a computation by such a Turing machine, oracle 
strings are (repeatedly) written to the unique write-only oracle tape and when 
the oracle consultation state is entered, whether the string written on the oracle 
tape is in the oracle (i.e., whether the string encodes a digraph and two vertices 
such that there is a Hamiltonian path in the digraph from the first vertex to the 
second) is reflected in the new state adopted by the machine. The oracle tape 
is cleared, with the oracle head subsequently positioned over the leftmost cell 
of the oracle tape; and all this happens in one move of the machine (see, for 
example, [1] for more on different types of oracle machines) . In comparison with 
the three points remarked upon above, note the following. 



A Perspective on Lindstrom Quantifiers and Oracles 



59 



(а) Whilst there are a number of different oracle access mechanisms in the li- 
terature (see, for example, S). it is usually the case that oracle calls can 
not be nested (and this is certainly the case for the standard notion of a 
log-space DOTM with an NP oracle). 

(б) The lengths of different oracle strings queried in a particular computation 
might vary dramatically. 

(c) The resources used to build a specific oracle query in such a computation 
can be log-space. 

So whilst there may be an intuitive link between Lindstrom quantifiers and 
oracles, there are a number of points of divergence. 



4 Characterizing Hamiltonian Path Logic 

As it happens, the points of divergence mentioned in the previous section can of- 
ten be reconciled. The complexity class consists of those problems accepted 
by a standard log-space DOTM with a CC oracle. 

Theorem 2. [33] 4 problem is in if, and only if, it can be defined by a 

sentence of the logic {ztHP)*[FOs\. Moreover, any problem in can be defined 
by a sentence of {±HP)*[FOs] of the form 

3zi3z2 ■ ■ . 3zfc(LfP[Axy(^](0, max) A -•HP[Xx'y' (j)']{0,Ynax)) 

where 4> and cf' are quantifier-free first-order {actually, quantifier-free projec- 
tions) and Zi, Z 2 , ■ ■ ■ , Zfc are free in (j) and </)'. 

Proof. (Sketch) Let 17 be some problem accepted by the log-space DOTM M 
with an NP oracle. Without loss of generality, we may assume that the oracle is 
HP (essentially because HP is complete for NP via 1-L reductions US]). Also, 
by a result of Wagner [40j . we may assume that on any input structure of size 
n, M makes O(logn) oracle queries. 

Consider a typical computation of M on some input structure of size n. 
There are some answers (0 or 1) to queries associated with this computation, 
and as there are O(logn) queries, we can encode these answers using the binary 
representations of k values vi,V 2 , ■ ■ ■ ,Vk S {0,1,... , n — 1}, for some k. If a is 
the underlying signature of 17, denote by a' the signature a with k additional 
constant symbols Ci,C 2 , - ■ ■ ,Ck- 

We shall define a log-space DOTM M' that takes cr'-structures as input and 
which accepts some cr'-structure A' if, and only if, M accepts A = A'\a (i.e., 
the cr-structure obtained from A' by removing the constants , (7^ , ■ . . , ) 

with the oracle answers as given by the constants , ■ ■ ■ , . 

The DOTM M' proceeds as follows. 

Simulate the action of M on M except that instead of writing on the 
oracle tape, use the bits of , (7^ , ■ . . , as the answers to (non- 
existent) queries. 
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If the simulation is rejecting then reject else: 

build, and query, a triple (G,a,b), where G is a digraph and a and b 
are distinct vertices of G, with the property that: if 

(Gi,Ui,6i),(G2,U2)^2)5-- - )(Gtj.j, CLjyi , bjji) 

are the queries made by M on input A for which the answer 
(according to , • ■ • , G^ ) is “yes” then (G, a, b) S HP if, and 

only if, (Gi,ai, 5 i), (G2, 02,62), .. . ,iGm,amAm) £ HP; 
build, and query, a triple (G', o', 6') with the property that: if 

(g;, o;, 6;), ■ ■ .,{G'^„a'^„bL') 

are the queries made by M on input A for which the answer 
(according to G^\C^' , . . . , G^') is “no” then (G', o', 6') ^ HP if, 
and only if, (G(, o'l, 6'i), (G'2, a'2, 6^, . . . , (G'„,, a(„,, 6(„,) ^ HP. 

If the answers to these two queries are “yes” and “no” , 
respectively, then accept otherwise reject. 



The triples (G, o, 6) and (G',o',6') can actually be built from (Gi, oi, 61), (G2, 
02,62),... ,{Gm,am,bm) and (Gi, o'l, 6'i), (G'2, O2, 63), . . . , (G'^,, o'^,, 6'^,) using 
1 -L reductions ED, and so M' is indeed a log-space DOTM. 

We can now encode the computation of M' so that there are two first- 
order quantifier-free (actually, quantifier-free projections) o'-formulae (/>(x, y) 
and <()'(x',y') with the property that 



A! ^ HP[Axyi^](0, max) if, and only if, there is a Hamiltonian path in 

the digraph G from o to 6 



and 

A! 1 = HP[Ax'y'(/)'](0, max) if, and only if, there is a Hamiltonian path in 

the digraph G' from o' to 6'. 

Consequently, fl can be defined by a sentence of the form 

3zi3z2 ■ • . 3zfe(HP[Axy<^](0, max) A -iHP[Ax'y'<))'](0, max)). 

Our only remark with regard to the encoding of the computation of M is 
that <^(x,y) and <^'(x',y') do not describe the digraphs G and G' directly. We 
actually describe these digraphs using formulae of (3iDTC)*[FOs] where the 
problem DTC is defined as 

DTC = {A G STRUCT ( (72-1-1- ) : the digraph with vertex set |Al| and 

edge set given by the relation E contains a deterministic path 
from vertex G to vertex £)}, 

with a deterministic path between two vertices in a digraph being one where every 
vertex on the path, except perhaps the last, has out-degree 1 in the digraph 
(Immerman captured L by the logic (3iDTC)*[FOs] [H])- We then apply the 
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operator HP to these formulae and manipulate the resulting formulae to get our 
normal form. 

The fact that any problem definable in (±HP)*[FOs] can be accepted by a 
log-space DOTM with an NP oracle is relatively straightforward to prove. □ 

The proof of Theorem [2] yields a number of corollaries regarding the rela- 
tionship between fragments of (±HP)*[FOs] and complexity classes defined by 
bounding the number of queries made by log-space and polynomial-time oracle 
machines to an NP oracle. We mention one of these corollaries below as an 
illustration (for others see |32l33| f . 

The Boolean Hierarchy, BH, is defined as follows. 

NP(0) = P; 

NP(1) = NP; 

NP(2t) = {X nY : X € NP(2i - 1), P e co-NP}; 

NP(2t -fi 1) = {a: u r : X e NP(2i), p e np}, 

for i > 0, and BH = U{NP(«) ; i = 0,1,...}. For any function /(n), 
L'^*’[0(/(n))] consists of those problems accepted by a log-space DOTM which 
makes 0(/(n)) queries to its NP oracle on every input of size n. For any logic L, 
the logic Bool{L) consists of the closure of L under A, V and Theorem |2] a sim- 

ple modification of its proof and a result in [10] (stating that if [0(1)] = 
then the Polynomial Hierarchy collapses to L ) yield the following corollary. 

Corollary 1. [23| Bool{HP*[FO s\) = L’^^[0(1)]; and so BH = L’^^[0(1)]. If 
Bool{HP*[FO s\) = {fsiF[P)*[FO s] then the Polynomial Hierarchy collapses to 

. □ 

The fact that BH = L'^^[0(1)] was proved in [23| . 

Write Lj^^[0(/(n))] to denote the complexity class consisting of all those 
problems accepted by log-space DOTMs which make, on an input of size n, 
0{f{n)) queries to an NP oracle such that all queries are made “in parallel”, 
i.e., intuitively, all queries are computed before any is made. As established in 
|40l . if either = Lj^^[0(log n)] or Lj^^[0(log n)j = L'^^[0(1)] then the 
Polynomial Hierarchy collapses. Thus, adhering to the current beliefs in comple- 
xity theory, it is unlikely that either of these two events happens. This prompts 
the question as to whether there is a logical characterization of the complexity 
class L|J^^[0(logn)] as a fragment of (±HP)*[FOg] (neither (±HP)*[FOsj nor 
Hool(HP*[FOsj) seem likely to suffice). 

5 Characterizations of Other Log-Space Oracle Classes 

Whilst Theorem |2] resolves the question posed earlier regarding identifying the 
logic (±HP)*[FOsj with a traditional complexity class, it is rather specific in 
that it deals only with the operator HP; and there is nothing particularly special 
about HP beyond the fact that it was involved in the first characterization of 
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NP by what one might call a “Lindstrom logic”. However, from Theorem [21 an 
analogous result holds for any operator for which the corresponding problem is 
complete for NP via quantifier-free first-order translations with successor (such 
as 3COL: see [30]). 

Such an observation led Gottlob to consider the more general question: “If 
12 is complete for some complexity class CC via first-order translations with 
successor, under what circumstances can we deduce that (±f?)*[POs] = 
and in |15| . Gottlob provided a precise answer which revolves around the con- 
cept of smoothness; where a complexity class CC is smooth if, and only if, 
L (CC) = L , i.e., the class of problems that are L many-one reducible 
to a problem in CC coincides with 

Theorem 3. [15] Let CC be some complexity class that is closed under log-space 
reductions, and let 12 be some problem that is complete for CC via first-order 
translations with successor. Then (±L?)*[POs] = L*^*^ if, and only if, CC is 
smooth. 

Proof. (Sketch) For simplicity, assume that f2 is over the signature (72 (the gene- 
ral case is similar). We begin by showing that if CC is smooth then any problem 
r in (±f?)*[FOs] is in We do this by induction on the length of the sen- 

tence defining P. The only non-trivial case is when this sentence <P is of the 
form 

f?[Axy^], 

where |x| = |y| = k, for some k, and where the predicate defined by ^(x,y) can 
be checked in 

Let A be some tT(T)-structure of size n. In order to ascertain whether A\^ 
we “build” a (72-structure B whose universe is |Al|^ and whose “binary relation” 
on \ A\^ contains the pair (u, v) if, and only if, 4>{u, v) holds in A. We then check 
to see whether this structure B is such that B £ f2. Thus, the problem P is 
many-one reducible to a problem in CC, i.e., P £ because CC is smooth. 

Now we show that if CC is smooth then any problem in L*^*^ can be defined 
by a sentence of (±l7)*[FOs]. In the proof of Theorem [2, we used specific pro- 
perties of the problem HP with regard to 1-L reductions in order to replace a 
number of oracle queries with just two oracle queries. However, we are now wor- 
king in a much more general context and, consequently, such problem-specific 
properties are not at our disposal. 

Let M be some log-space DOTM with a CC oracle that accepts the problem 
P. Consider a computation of M on the (7(7”) -structure A of size n. We may 
clearly assume that no instantaneous description (ID) occurring in a computation 
of M is ever repeated in this computation (for us, an ID of M consists of the 
state, the contents of the work-tape, the work-head position and the input-head 
position at any particular instant of a computation of M) and that whenever M 
terminates (which it does on every input), the oracle tape is empty. Note that 
because M is a log-space DOTM we can: 
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• encode an ID of M as fc values vi,V 2 , - ■ ■ ,Vk G {0, 1, . . . , n — 1}, for some k; 

• “decode” a given /c-tuple u G {0,1,... , n — 1}^ so as to decide in log-space 
whether it really encodes a legitimate ID of M on some input structure of 
size n. 

Henceforth, we assume that all fc-tuples over (0, 1, . . . ,n — 1} encode potential 
IDs in some computation of M on an input structure of size n; as, by above, this 
property can be checked in log-space. 

Let u and v be IDs of M on input A such that when M is started in ID u, 
the next ID it reaches where the oracle tape is empty is v. Then we say that v is 
the direct successor of u. Additionally, if there is no oracle computation between 
u and V, we say that v is the strict successor of u. 

Consider the following two problems. 

Problem A Given two IDs u and v of M and the input structure A, u and v 
are such that either: v is the strict successor of u; or v is the direct successor 
of u, there is an oracle query between u and v and the answer to this oracle 
query is “yes”. 

Problem B Given two IDs u and v of M and the input structure A, v is 
the direct successor of u, there is an oracle query between u and v and the 
answer to this oracle query is “no” . 

Then Problem A is log-space reducible to a problem in CC, and Problem B is 
log-space reducible to a problem in co-CC. Thus, by hypothesis. Problems A 
and B can be defined by cr(T)-formulae of (±17)* [FO^] of the form 

G[Axy()i(x,y,z,w)] and ,y' ,z,w)], 

respectively, where (p and (j)' are formulae of FOg (the tuples of free variables 
z and w encode the IDs). Consequently, there is a cr(T)-formula <P{z,w) of 
(±G)*[FOs] of the form 



l7[Axy</)] A -^f![Xx.'y' (p'] 

which holds in some structure A for some IDs u and v if, and only if, v is the 
direct successor of u in the computation of M on input A. 

We can now use the operator DTC and our formula ^(z,w) to define the 
problem consisting of all those cr(T)-structures A such that on input A, there 
exists an accepting ID that can be reached from the initial ID in the digraph 
defined by <?(z, w); in other words, we can define the problem F. However, as L 
C CC, we can eliminate any applications of the operator DTC at the expense 
of introducing applications of the operator 17. Hence, T can be defined by a 
sentence of the logic (±l7)*[FOg] as required (and the defining sentence just 
constructed is actually in (±l7)^[FOs]). 

In order to prove our theorem, we need to show that if (±l7)*[FOg] = 
then CC is smooth. Suppose that (±l7)*[FOg] = Let the problem F 

be many-one reducible to the problem 6> G CC. By hypothesis, this L*^*^ 
many-one reduction can be defined by a formula of (±l7)*[FOg], and the problem 
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0 can be defined by a sentence of (±i?)*[FOs]. Hence F can be defined by a 
sentence of (±t7)*[F0g], and so CC is smooth. □ 

Whilst Theorem 13 gives an answer to the question posed at the beginning of 
this section, it also provokes the question: “How can we tell whether a complexity 
class is smooth or notT' . Indeed, for all we know all complexity classes might be 
smooth. The most transparently smooth complexity class is L as it is well known 
that the composition of two log-space machines can be replaced with a log-space 
machine. Essentially, whenever the second machine (in the concatenation) needs 
some input bit, it computes the input bit required, using log-space, and proceeds 
accordingly. The important point to note is that it need not “remember” the 
whole of the output from the first machine (of the concatenation), which might 
be of polynomial size. 

If we were to naively attempt to replicate this construction with the first 
machine. Mi, a L*^*^ machine and the second machine, M 2 , a CC machine, so 
as to replace the two machines with one machine, M, then (intuitively) 
we run into problems. Suppose that M is in the process of building some oracle 
string to query, but half way through this building process it discovers that it 
needs to compute some input bit to its simulation of M 2 , i.e., an output bit 
from Ml. Naively, it would perform a simulation of Mi; but this might not be 
possible as Mi might wish to query some oracle strings in this computation, and 
the oracle tape of M is blocked with a half-built query already! In fact, there do 
exist complexity classes which are closed under log-space reductions and which 
are not smooth although we do not prove this fact here (but simply refer the 
reader to m)- 

Having satisfied ourselves that not all (reasonable) complexity classes are 
smooth, let us look at some complexity classes which are smooth; and, in par- 
ticular, at some criteria for smoothness. Many complexity classes (especially 
deterministic ones) can easily shown to be smooth simply by working through 
the definition of smoothness with the particular complexity class: such classes 
include L, POLYLOG SPACE and P. Other complexity classes can easily be 
shown to be smooth by applying the following lemma; but first two definitions. 
Reverting back to the traditional definition of a complexity class as a class of 
languages, we say that a complexity class CC is closed under marked union if for 
every two languages Ai and A 2 in CC, the marked union of Hi and A 2 , namely 
the language obtained by inserting a 0 before every string of Hi and a 1 before 
every string of H 2 , and then taking the union of the resulting two languages, is 
also in CC. The complexity class P*^*^ consists of those problems accepted by 
a standard polynomial-time DOTM with a CC oracle. 

Lemma 1. |15) IfP^^ = CC and CC is closed under marked union then CC 
is smooth. □ 

The complexity classes Af, for i > 1, of the Polynomial Hierarchy, and the 
complexity classes PSPACE and fc-EXPTIME, for fc > 1, can be shown to be 
smooth by applying Lemma [T] 
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Other sufficient criteria for smoothness exist. A complexity class is closed 
under conjunctions if, and only if, for any finite set of instances of some problem 
r G CC, the problem of deciding whether every one of these instances is in F 
is also in CC. A problem 17 is 'NP-reducible to some problem F if, and only if, 
there is a non-deterministic polynomial-time transducer M such that on input 
some (r(l7)-structure A, there is at least one computation of M on A which 
outputs a cr(/^)-structure B such that B G F. 

Lemma 2. | 15| If the complexity class CC is closed under BiP -reductions, un- 
der conjunctions and under marked union then CC is smooth. □ 

The complexity classes Sf and Ilf, for i > 1, of the Polynomial Hierarchy, 
and the complexity classes NEXPTIME and co-NEXPTIME can be shown 
to be smooth by applying Lemma |2] 

The criteria for smoothness above, whilst widely applicable, do not seem to be 
strong enough to show that every smooth complexity class is in fact smooth. The 
most notable example is the complexity class NL for which smoothness follows 
from results due to Immerman and for which neither of Lemmas [D nor E] 
seem to be of any use. It would be interesting to derive other criteria from which 
the smoothness of (complexity classes such as) NL could be deduced. 

Let us end this section by remarking that the proof of Theorem [2] also yiel- 
ded a normal form result, whereas no analogous normal form result has been 
forthcoming from the proof of Theorem [3l Perusal of the proof of Theorem 
yields that the completeness of HP via 1-L reductions played a role; and this 
might imply that normal form results could be specific to particular problems. 
However, Gottlob has established general criteria for determining when a normal 
form result, analogous to that in Theorem!^ exists. 

Theorem 4. | 15| Let CC be some complexity class that is closed under NP- 
reductions, conjunctions and marked union, and let fl be some problem that is 
complete for CC via first-order translations with successor. Then every problem 
in = (±f7)*[EOs] ean be defined by a sentence of the form 



3zi3z 2 . . . 3zk{^ A 



where F and <P' are formulae of Q^fFOg] formed by applying the operator 12 to 
appropriate first-order formulae and where zi,Z 2 , - ■ ■ ,Zk are free in F and F' .U 

The proof of Theorem [H is similar to the proof of Theorem E] except that 
it uses the fact that a complexity class closed under conjunctions and NP- 
reductions is necessarily closed under conjunctions and disjunctions (defined 
similarly) to obviate the need for the oracle to be complete for some complexity 
class via 1-L reductions. It would be interesting to investigate whether there is 
any sort of relationship between a complexity class having a complete problem 
via 1-L reductions and a complexity class being closed under operations such as 
those in Theorem E] 
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6 In the Absence of Successor 

As remarked earlier, the motivating theorem for the thread of research presented 
in this paper, Theorem [l] holds in the absence of the built-in successor relation. 
Consequently, it is natural to ask whether the same can be said of the subsequent 
results; in particular. Theorem Dawar, Gottlob and Hella have given partial 
answers in this regard, as we now explain; but first we require some definitions. 
We denote by: 

• the fragment of FO which consists of those formulae whose variables, 
both bound and free, are among xi,X 2 , ■ ■ ■ ,Xk', 

• the closure of under the operations of conjunction and disjunc- 
tion applied to arbitrary (finite and infinite) sets of formulae (with obvious 
semantics); 

• the union 

The logic was introduced by Barwise and is known as bounded-variable 
infinitary logic. It plays an important role in finite model theory (see, for exam- 
ple, [9]). 

However, we introduce here for a specific reason. Bounded- variable infi- 
nitary logic can be extended by a set of Lindstrom quantifiers fl (possibly infinite 
and not necessarily in the form of uniform sequences corresponding to problems) , 
just as FO was (and with similar semantics), to yield the logic 
Dawar and Hella jH] have established certain properties of logics of the form 
or more specifically, of fragments of such logics where the num- 
ber of quantifiers of 17 appearing in any formula is finite, which are of direct 
relevance to the investigations presented here. 

Let A be some structure and let u = {ui,U 2 ,-.. ,Uk) G \A\^ . The basic 
equality type of u is the formula 

f\ {x, = Xj)A f\ i^{xi = Xj)), 



where S = {{i,j) ■ i < j and Ui = Uj} and T = {{i,j) '■ i < j and Ui yf Uj}. 

Let ^ be a formula of (±17)* where 17 is a set of Lindstrom quanti- 

fiers. Then ^ is a basic flat formula if it is atomic or if it is formed by applying 
a quantifier Q S 12 to some first-order quantifier-free formulae. Also, (j) is in flat 
normal form if it is obtained from basic flat formulae by successive applicati- 
ons of the connectives V, A and -■ and the first-order quantifiers V and 3. For 
example, the formula HP[Axyc;ii](0, max) of Theorem |5] is a basic flat formula, 
whereas the formula 3zi3z2 ■ ■ ■ 3zfe(HP[Axy(;ii](0, max) A-'HP[Ax'y'i;i'](0, max)) 
of Theorem [ 2 ] is in flat normal form. 

We are now in a position to state Dawar and Hella’s result (which although 
stated in is proved in i)- 

Theorem 5. m Let (Te be the empty signature, and denote the unique struc- 
ture of size n over (le by An- Let 17o be a finite set of Lindstrom quantifiers. 
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(a) For every n, there exists a sentence ijn G (±l?o)*[-^^] such that for every 
n' , An' H Vn if, only if, An and An' satisfy the same sentences of 

{±Qor[LtJ- 

(&) The sentence rjn is of the form 

3xi . . .3xktpi A\/xi . . .Vxk \f ift A f\ \/xi . . .yxk{4>3 ^ Ij), 

l<i<m l<2<m 

where ■i/'i , V^ 2 , ■ • ■ ,V'm basic equality types of k -tuples of An, each 

formula is a disjunction of some basic equality types and each formula (pj 
is a basic fiat formula. 

(c) Given some u G there is a formula r]n,u of the form 



77„ A Va:i+i . . .Va:fe7, 



where j is a disjunction of some basic equality types, such that for every Aw 
and u' e 

Ain' 1= Vn,u(u') if, ond only if, the expansions (^„,u) and {An',vA) 
satisfy exactly the same sentences of {3zQif)*\L^J\. 



□ 



Corollary 2. [Zj Let Gq be a finite set of Lindstrom quantifiers and let be 
the empty signature. For every formula (p{xi,X 2 , ■ . ■ ,xi) G (±J7o)*[^Sotj(o’e)], 
there exists a formula ip{xi,X 2 , ■ • ■ ,xi) G (±17o)^[^"0(ctj)] such that (f> and 'ip are 
equivalent. 

Proof. There is clearly some k such that p G {3zGQ)*[L’j^^{af)]. Fix n and let 
ipi,'il) 2 , ■ ■ ■ , 1pm be the basic equality types of fc-tuples of elements of An- Let ? 7 „ 
be the sentence as in the statement of Theorem [SI and for every u G let 

r/n,u be as in Theorem[5l Define F = {(n,u) : u G \An\’’ such that An |= </>(u)} 
and define ip as 

\f 1ln,\i- 

(n,u)£F 

Consider any n and u G \An\^ ■ Suppose that An |= So, (n,u) G F, r]n,u 

is a disjunct of ip and An |= 7 n,u(u). Thus, An H V'(u)- Conversely, suppose 
that An 1= ip{A). So, An 1= ? 7 ti',u'(u), for some (n',u') G F, with the result that 
(^„,u) and {An','»!) satisfy the same sentences of (±J7o)*[L^,^]. In particular. 
An 1= The result follows. □ 

The following is immediate from Corollary E] 

Corollary 3. [7j //17 zs o set of Lindstrom quantifiers such that every quantifier 
can be evaluated in NP then every sentence of (3zG)*[FO{af)] is equivalent to 
a Boolean combination of NP properties. □ 
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Given a language A over {0,1}, 

tally (A) = |1” : the binary representation of n G lA}. 

The Linear Exponential Boolean Hierarchy, EBH, is defined as follows. 

EBH(O) = DETIME = DTIME(2<^(")); 

EBH(l) = NETIME = NTIME(2'^(")); 

EBH(2i) = {X r\Y X e EBH(2t - 1), T G co-NETIME}; 

EBH(2i +1) = {XUY : X G EBH(2t), T G NETIME}, 

for i > 0, and EBH = U|EBH(t) : i = 0, 1, . . . }. The Full Exponential Boolean 
Hierarchy, EXPBH, is defined as follows. 

EXPBH(O) = DEXPTIME = DTIME(2'^(p°'?^("))); 

EXPBH(l) = NEXPTIME = NTIME(2 <^(p°'«("))); 

EXPBH(2t) = {X DY : X G EXPBH(2t - 1), F G co-NEXPTIME}; 
EXPBH(2t +1) = {XUY : X G EXPBH(2i), F G NEXPTIME}, 

for i > 0, and EXPBH = U{EXPBH(t) : t = 0, 1, . . . }. 

The hierarchies EBH and EXPBH can be regarded as exponential ver- 
sions of the Boolean Hierarchy. Indeed, we can go further and associate with 
many complexity classes CC contained in the Polynomial Hierarchy, a linear 
exponential version E(CC) and a, full exponential version EXP{CC). Such an 
association is detailed in Fig. 1 below, with the motivation behind such an asso- 
ciation given in the subsequent lemma. The definitions of the complexity classes 
have either been given or are obvious, except that {F?, 77} : z = 0, 1, . . . } (resp. 
j^^expjjexp ■ j = 0, 1, . . . }) are the classes of the Linear (resp. Full) Exponential 
Hierarchy EH (resp. EXPH), built around ETIME and NETIME (resp. EX- 
PTIME and NEXPTIME) just as the Polynomial Hierarchy is built around 
P and NP. 



Basic complexity class 

CC 


Linear exponential version 
E{CC) 


Full exponential version 
EXP{CC) 


P 


ETIME 


EXPTIME 


NP 


NETIME 


NEXPTIME 






^exp 


nf 


nf 


jj^exp 


PH 


EH 


EXPH 


BH(7) 


EBH(7) 


EXPBH(t) 


BH 


EBH 


EXPBH 


L 


LINSPACE 


PSPACE 


NL 


NLINSPACE 


PSPACE 


lNP 


LINSPACE^^ 


pspace’^^ 



Figure 1. Linear and full exponential versions of complexity classes. 
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Lemma 3. |16| For each basic complexity class CC in the first column of Fi- 
gure 1: 

• the closure under polynomial-time many-one reductions of E{CC) is the 
complexity class EXP{CC); and 

• for every language A G CC, tally(A) G CC if and only if, A G E(CC). 

□ 



We are now ready to establish the results of Dawar, Gottlob and Hella. 

Theorem 6. [Zj If there exists a family 12 of Lindstrom quantifiers where each 
quantifier in f2 can be evaluated in NP and where (±17)* [PO] = then: 

• EBH = LINSPACE’^*’ and EBH collapses; and 

• EXPBH = PSPACE^** and EXPBH collapses. 

Proof. Let A be a language in LINSPACE'^^. By Lemma tally(A) G 
Define the problem F over to be such that tally(A) is the natural encoding 
of F. By hypothesis, F G (±17)* [FO] = and so by Corollary [3l 

tally(A) G EBHfe, for some k. 

But LINSPACE'^^ has complete problems, and consequently LINSPACE'^^ 
C EBHfc. As EBH C LINSPACE'^*’, the first part of the theorem follows. 

However, the second part of the theorem also follows as, by Lemma O the 
complexity classes EXPBHfc, EXPBH and PSPACE'^^ are the closures of 
the complexity classes EBHfc, EBH and LINSPACE'^^, respectively. □ 

Any collapses of the form detailed in Theorem would cause not inconsi- 
derable surprise amongst many complexity theorists. Therefore, we are led to 
speculate that the complexity class can not be captured on the class of 
all structures by a “Lindstrom logic” . It would be extremely interesting if this 
result could be established. However, a great improvement of Theorem would 
be if the premise in the theorem yielded a complexity-theoretic collapse “lower 
down” , say of the Polynomial Hierarchy. 

We close by noting that things are different for some complexity classes 
“beyond” P'^*’. Amalgamating results due to Gurevich m and Blass and Gu- 
revich 1^, Dawar [6] observed that any recursively presented complexity class 
containing p'^^ and closed under compositions is recursively indexable, and so 
there is a logic capturing this class. Other results in pj then yield that if the 
complexity class is bounded (see m), then it can be captured by a “Lindstrom 

y^p 

logic”. So, for example, L = can be captured by a “Lindstrom logic” (as it is 
bounded) . 
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Abstract. Generalized quantifiers are an important concept in model- 
theoretic logic which has applications in different fields such as lingui- 
stics, philosophical logic and computer science. In this paper, we consi- 
der a novel application in the field of logic programming, which has been 
presented recently. The enhancement of logic programs by generalized 
quantifiers is a convenient tool for interfacing extra-logical functions and 
provides a natural framework for the dehnition of modular logic pro- 
grams. We survey the expressive capability of syntactical classes of lo- 
gic programs with generalized quantifiers over finite structures, and pay 
particular attention to modular logic programs. Moreover, we study the 
complexity of such programs. It appears that modular logic programming 
has the expressive power of second-order logic and captures the polyno- 
mial hierarchy, and different natural syntactical fragments capture the 
classes therein. The program complexity parallels the expressive power in 
the weak exponential hierarchy. Modular logic programming proves to be 
a rich formalism whose expressiveness and complexity can be controlled 
by efficiently recognizable syntactic restrictions. 



1 Introduction 

Generalized quantifiers are a well-known concept for enhancing the expressive 
capabilities of a logical language |44I34I7I59| . They have been introduced since 
in many contexts, the standard quantifiers “for all individuals” and “for some 
individual” are not strong enough for a proper description of the state of affairs. 
Informally, a generalized quantifier Qn. is a collection C of structures for a lan- 
guage C, and Qx(/?(x, y) evaluates on a structure 21 to true for elements y G H, 
if the structure (A, {x G A | 21 ^ (/?(x,y)}) belongs to C. 
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In 1957, Mostowski [44] initiated the field, and stimulated a lot of work on 
generalized quantifiers during the 60’s and early 70’s. Another approach to gene- 
ralized quantifiers from 1959 are Henkin quantifiers, which have been introduced 
in m and subsequently studied in numerous papers, e.g. |18I64I5I9I33(50| . Unify- 
ing the two approaches, the most prominent work from this area is the seminal 
paper of Lindstrom |34| , whose formalization of generalized quantifiers is mostly 
used to date. After diminished attention for some time, generalized quantifiers 
gained in the 90’s increasing interest in applications, from which new research 
issues emerge. The applications of generalized quantifiers range over different 
fields, including linguistics, philosophical logic and computer science. 



In linguistics and philosophical logic, it has been recognized that the quan- 
tification theory of first-order logic is insufficient as a representation formalism 
for natural language statements, cf. |25]. Montague [l^ made an attempt to 
properly treat quantifier phrases in English, the idea to which, as pointed out 
in m, originally dates back to Russell jlSj . Building on the work of Montague 
|43j . Barwise and Cooper have presented an approach which adopts generalized 
quantifiers in a natural language formalization [^. 



In computer science, generalized quantifiers are used in different areas. On 
the foundational side, they have been considered extensively in descriptive com- 
plexity theory as a tool for designing logical languages for capturing compu- 
tational complexity classes. For example, restricted Henkin quantifiers capture 
NL [Hj over ordered structures, while standard Henkin quantifiers capture 
over ordered structures [221; transitive closure logic captures NL over ordered 
structures, while deterministic transitive closure logic captures L over ordered 
structures isnES]; and, various extensions of first-order logic by quantifiers such 
as the Hamiltonian Path Quantifier capture NP or (i.e., logspace with an 
NP oracle) over ordered structures, [53154122] . The relationship of oracles, gene- 
ralized quantifiers and subprograms was also investigated in [3SI37I40I41J . See 
pmrn] for a comprehensive overview. Notice that all syntactic complexity classes 
can be captured by generic generalized quantifiers |3fil39lfi1j . 

In the context of databases, generalized quantifiers have been used as a me- 
ans for increasing the expressive capability of database query languages. In [J, 
the transitive closure quantifier was applied ad hoc since connectivity cannot be 
expressed in Codd’s relational calculus. A more systematic treatment is given in 
mn . where a formalization of subqueries in SQL by generalized quantifiers is 
presented, and occurrences of generalized quantifiers in existing database query 
languages are discussed. Further applications of generalized quantifiers in com- 
puter science are in software engineering, where generalized quantifiers are used 
in the specifications of the Z and B notation [52|fi5| . Moreover, generalized quan- 
tifiers have also been investigated in the context of fuzzy logic m, which has 
become popular in computer science more recently. 



In this paper, we address another application of generalized quantifiers in 
computer science; namely, in the field of logic programming. As pointed out in 
m, enhancing logic programming by generalized quantifiers provides an ele- 
gant and appealing method for handling the following two problems. Firstly, 
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the problem of interfacing external (non-logical) functions in a logic program, 
of which no other information than a semantical description is available. Such a 
situation is present e.g. with library routines, for which usually only a functional 
specification is known. Secondly, the problem of modular programming, where 
a formalism for the synthesis of program modules into logic programs is sought. 

The two problems are not unrelated; the latter problem can be seen as an 
instance of the former, in which modules are functions for which information 
about the implementation in terms of proof theory etc. is available. For both 
problems, so far mainly ad hoc solutions have been proposed. Modular logic 
programming has been approached from two main directions |10| . The one di- 
rection is to build an algebra of programs, in which composition operators are 
available for combining two programs into a single one (cf. 1191451 ). The other 
direction resorts to using concepts of modal logic and embedded implications for 
defining the meaning of subgoals which access external functions (cf. |l2]). Both 
approaches, however, have problems in handling negation, and typically, nega- 
tion is disregarded for combined programs. This is a severe shortcoming, since 
negation is an indispensable element of a logic-based programming language to 
be sufficiently expressive for solving problems in practice. 

Our extension of logic programs by generalized quantifiers is a natural en- 
richment of the formalism which allows for a natural and appealing solution 
to the problem of incorporating external functions into logic programs. As an 
application, we obtain a semantics for modular logic programming which is con- 
ceptually clean and, as a major advantage, harbors a full treatment of negation 
in the style of ordinary logic programs with negation. 

Roughly, the idea of the approach is to allow in the bodies of rules besides 
standard logical literals also formulas (5x[i?(x)], which describe the application of 
a generalized quantifier Qx to a structure defined by the predicate R. Informally, 
Qx can been seen as an interface to an external function whose argument is the 
predicate R. For any extension of R, the function resp. quantifier Qx evaluates 
to either true or false, and thus succeeds or fails as a subgoal in a rule body. An 
extension to functions with multiple arguments is straightforward. 

In the present paper, we survey the expressive capability of logic programs 
with different classes of generalized quantifiers, where we focus mainly on finite 
structures. We pay particular attention to various natural syntactic fragments 
of the formalism. The programs may use generalized quantifiers of a particular 
complexity, and in particular generalized quantifiers which are definable by logic 
programs. 

It appears that normal logic programs with quantifiers in NP capture the 
class TTf of the polynomial hierarchy; thus, by adding such quantifiers, the ex- 
pressive capability of normal programs increases from co-NP m by one level in 
the hierarchy. As a consequence, modular logic programs capture as well. The 
expressive capability further increases by allowing nested program modules. The 
natural extension of the formalism to nested modular logic programs captures 
the polynomial hierarchy, and is therefore as expressive as second-order logic over 
finite structures. In fact, the equivalence of nested modular logic programs to 
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second order logic holds even for arbitrary structures. Moreover, natural syntac- 
tic fragments of the formalism capture the classes 77^, A: > 1, of the polynomial 
hierarchy, and over ordered finite structures, also the classes and 0^, for 
A; > 1. 

The rest of this paper is organized as follows. In the next section, we recall 
some basic concepts and fix notation. In Section [3l we consider logic programs 
with generalized quantifiers, and in Section [d] we describe a formalism for mo- 
dular logic programming based on logic programs with generalized quantifiers. 
Thereafter, we address the expressive capability of the formalism in Section |5] 
and the computational complexity in Section E Programs with nested modules 
are considered in Section 0 Finally, Section [HI concludes the paper. 

2 Preliminaries and Notation 



Notation. Letters P,Q,.. . denote predicates, lower case letters x, y, z variables, 
a,b,c, . . . constants and f,g... functions. The bold face version P of a predi- 
cate symbol P denotes a list P±, ■ ■ ■ , Pm of predicate symbols, and similar for 
variable, function, and constant symbols. Fraktur letters 21, *8, . . . denote logical 
structures. Sets of structures are denoted by capital Greek letters 77, S or C, 
and classes or mappings thereof by A7, Q, . . . ; lower case Greek letters r,a,. ■ ■ 
denote signatures. 

Logic. A signature!!] is a sequence r = (P“^ , . . . , P ^'‘ , /;fb . . . , , ci, . . . , Cm) 

where the Pi are relational symbols of arity > 0, the fi are functions with 
bi > 1 arguments, and the Ci are constants, r is relational, if it contains only 
relational symbols. 

A structure 21 over r is denoted by (A, P®, . . . , P®, /®, . . . , /®, cf , . . . , c® ). 
A is called the universe or domain of 21, and denoted |21|. 21 is finite if |21| is 
finite. The set of all structures over r is denoted by Struct(r). 

Let 21, IB G Struct(r), s.t. |2l| = |1B|. Then 21 C IB, if P® C P-® for 1 < i < A:, 
ff' = /® for 1 < J < Z, and cf = c® for 1 < i < m. A computational problem 
over signature r is a set P C Struct(r), s.t. II is closed under isomorphisms. II 
is monotone, if 21 C 05 and 21 G P implies 05 G P. 

For a relational signature r and integer I, = (P;[“^ , . . . , Pj ^'° ) is called the 
I— ary vectorization of r. For a computational problem P, let denote the 
problem P over Z— tuples, i.e. over universe n x ■ ■ ■ x n|l Let 21 be a relational 

I times 

structure, and U C |21|. Then the restriction of 21 to U, in symbols 21|P, is the 
structure {U, P® fl , . . . , P® fl P“'=). For a r-structure 21 and a signature Tq 

^ In contrast to the usual convention in finite model theory, we do not disregard 
function symbols in our language at this stage, because they are commonly available 
in logic programming. 

^ This is a coherent definition, since we deal only with problems which are closed 
under isomorphisms. 
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contained in r, 2t|ro is the rg-structure obtained from 21 by removing all relations, 
functions, and constants not contained in tq. 

The set of all finite models of a formula S' is denoted by Mod(if'). 

Let (f(xi , . . . , Xn) be a formula with free variables xi, . . . , and let 21 be a 
structure. Then denotes the n— ary relation {(di, . . . , dn) \ 21 \= . . . , d„)}. 

Let £ be a syntactic fragment of first-order logic. Given signatures r, a and 
a natural number k, a k-axy interpretation / of r into cr is a definition of the 
relations in terms of t, i.e., a tuple of C formulas, s.t. for each predicate 
symbol i? in cr with arity r, I contains a formula (pn over t with rk free variables 
which defines For a structure 21 G Struct(r), /(2l) denotes the structure over 
cr^^l which is defined by I. Let U C Struct(r), S C Struct(cr) be problems. II 
is £-reducible to S if there exist an interpretation / of r into a, s.t. for all 
21 G Struct(r), 21 G 7T iff /(2l) G where k is the arity of I. By restricting the 
logic C for the interpretations, we obtain low-level reductions: A quantifier-free 
reduction I3D] is a reduction whose defining formulas are quantifier-free. 

Logic Programs. A logic program is a finite collection of clauses (also called 
rules) of the form 



A •<— i?i, . . . , Bjn (1) 

where A is an atom and the Bfs are literals in a first-order language. We use 
-■ for the negation (as failure) operator in logic programming. A program is 
positive, if no negation occurs in it. 

The meaning of a positive logic program P has been defined in terms of its 
least model; for logic programming, a number of different semantics have been 
proposed, since the definition of negation is nontrivial and allows for alternative 
approaches, cf. m- For background, consult [d5] . 

Example 1. In the following program, the extension of TC in the least fixpoint is 
the transitive closure of the binary relation E. We say that the program computes 
TC. 



TC{x,y) ^ E{x,y) 

TC{x,y) ^ TC{x,z),E{z,y) 



Generalized Quantifiers (GQs). Let C be a class of logical structures over a 
relational signature a = {Ri, . . . , i?„) with arities oi, . . . , a„, and suppose that 
C is closed under isomorphism, i.e., if 21 = iB and 21 G C, then *8 G C. Each 
such class C has an associated GQ Qc, whose intended semantics is to check if 
a relation defined by the underlying logic belongs to dH 

® In logic programming, closedness under isomorphism assures that terms are treated 
as completely uninterpreted by the GQ. If only particular interpretations are consi- 
dered, e.g., Herbrand interpretations, weaker closure conditions are in order; cf. the 
discussion about genericity in logical database query languages. 
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The extension C{Qc) of a logic £ by a GQ Qc is the closure of C under the 
following rule: 

If :^i(xi), . . . , (pn(xn) are formulas of L{Qc) where every has at least 
free variables x^, then ■ ■ • x„[(/ji, . . . , </?„] is a formula of L{Qc), in which 

the occurrences of xi, . . . ,x„ are bound. For clarity, we shall often write the list 
of remaining free variables y after the formula. 

The semantics of Qc is defined as follows. Consider the formula 

0 = Qc^i ■ ■ . . . ,^Pn]{y), 

and let 21 be a structure and b a tuple in 21 matching the arity of y. Then 
(21, b) \= 0, if and only if the structure (A, belongs to C, 

where = {a | 21 ^ b]}. The family C of C -quantifiers is the set 

{Qc, Qc(2), Qc(3), • ■ • }• 

Example 2. The following GQs will be used in further examples. 



1. The complement quantifier: 

2. The majority quantifier: 

3. The modularity quantifier: 

4. The isomorphism quantifier: 

5. The transitive closure quantifier: 



={{A,U,{v})\veA-U} 

Qm ={{A,U,V) I U,VCA,\U\ > |F|}. 

Qk ={{A,U) I |C/| =0 (modfc)}. 

Qs ={{A,E,F) I (A,E), 

{A,F) are isomorphic graphs}. 

Qtc= { {A^ E, |(u, u)}) I there is a path 

from It to in the graph E C A x A}. 



Complexity Theory. We assume some familiarity with the basic concepts of 
complexity theory, such as the classes P, NP, L (logspace), oracle computations, 
and the polynomial hierarchy (PH) whose classes are and A^, k > 1; 

consult for background. 

Intertwined with the levels 11^ of PH are the classes {S^ C 

C which are now considered intermediate constituents of PH. The 

0k+i classes have natural complete problems in terms of optimization variants 
of i7(!-complete problems. 

The class Pj^^ is the class of problems decidable by polynomial time Turing 
machines whose oracle queries are first collected, and then all queries are for- 
warded to the oracle together. Thus, the queries are independent of each other. 

Proposition 1. [B3| 0^+1 = 

3 Logic Programs with Generalized Quantifiers 

Let To be a signature for describing the program input, and let r be an extension 
of To by new relational symbols. A logic program with GQs (GQLP) on t is a 
finite collection V of rules 



A •<— Hi , . . . , H, 



( 2 ) 
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where A is a r-atom whose predicate does not occur in tq, and each Bi is either 
a r-literal or a generalized quantifier literal (GQ-literal) over r, which is defined 
next. 

Suppose that Qc is a GQ defined over the signature a = (i?i, . . . , i?„, Rn+i) 
with associated arities ai, . . . , a„, a„+i = /, and that Si, . . . , Sn are predicates 
from r such that the arity of Si equals Oi. Then, the formula 



QcXi---x„+i[S'i(xi),...,S'„(x„),x„+i = v](v) (3) 

is a GQ-atom. Notice that the free variables of this GQ-atom are v = ui, . . . ,vi. 
The formula x„+i = v defines the singleton relation {v}; intuitively, it transfers 
domain elements into the quantifier. A GQ-literal is a possibly negated GQ-atom. 
For brevity, we denote a GQ atom © by 



Qc[S](v) (resp. Qc[S], if v is void) (4) 

where S = S'!, . . . , Sn, and similar for negative GQ-literals. 

For any collection Q of generalized quantifiers, we denote by Tq the extension 
of T by all predicate letters Qc [S] , where Qc S Q and S = S'! , . . . , S'n is a list 
of predicate letters Si from r which matches the signature of Qc', every such 
QclS] is a GQ-predicate. Notice that Tq is finite if r and Q are finite. Then, a 
GQLP over t is syntactically an ordinary logic program over the signature Tq. 

Example 3. Gonsider the following program V, which uses two GQs: The isomor- 
phism GQ Qs [Gi , G 2 ] , which tells whether Gi and G 2 are isomorphic graphs, 
and the transitive closure GQ TG: 

S{x,y) ^ QTc[E]{x,y),QTc[E]{y,x) 

Ga{x,y) ^ E{x,y),S{a,x),S{a,y) 

Gb{x,y) ^ E{x,y),S{b,x),S{b,y) 

Iso ■<— Qsi[Ga,Gb] 

Suppose that tq contains the relation E and the constant symbols a and b. Given 
a graph G = {V, E), and vertices a, 5 G V, this program assigns the propositional 
letter Iso true on E if the strongly connected components in which a and b lie 
are isomorphic. 

Some remarks about the use of GQs as proposed above are in order. In first- 
order logic, the scope of a GQ Qc^i ■ ■ ■ Xn[(^i, . . . , ipnliy) may contain arbitrary 
first-order formulas cpi and also GQs. In the context of logic programming, there 
is no need for allowing complex argument formulas pi, since such a pi can be 
associated with a defining equivalent auxiliary relation A,p. which is computed 
within the logic program. Moreover, the restriction of free variables to the for- 
mula y = V is not severe; roughly, free variables z in a formula (/?(xi,z) can be 
treated as bound and newly introduced free variables z' (put in v) can be used 
to single out the relation p^ for the semantical definition of the GQ. 
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In the context of first-order logic, the evaluation of formulas with GQs can be 
easily defined by structural induction. However, in logic programming, we have 
to deal with recursion, which makes the definition of a semantics nontrivial, even 
if no GQs are present, but just negation. 

Since negation is provided by the family of complement GQs Qr^, any seman- 
tics of positive logic programs with GQs implicitly provides a semantics for logic 
programs with negation. Since there is no general consensus about the semantics 
of a logic program with negation, a generally accepted semantics for GQLPs may 
not be expected. 

We define the semantics of a GQLP V in the spirit of the stable model seman- 
tics for logic programs with negation m, which is widely accepted. Informally, 
a stable model of a program V amounts to a solution of a fixpoint equation 

where /(7^,9Jt) is the result of V if negation is evaluated with respect to 3H. 

Our approach is to treat GQ-literals like negative literals; since GQs subsume 
negation, negation is implicitly treated with (unnegated) GQ-atoms. Under an 
assumption, based on OJt, about the truth of GQ-literals and negative standard 
literals, a GQLP V is reduced similar as in to an ordinary positive logic 
program which is evaluated. If the least minimal model of the latter program 
justifies the assumptions made by Tl, it is considered an acceptable model of V', 
we call such a model GQ-stable. 

We next provide a formalization of this concept. Suppose that we have sig- 
natures To and T as above, a GQLP V and a structure 2t S Struct (t). 

Then, the ground instantiation of V on 21, denoted ground(V,^), is the 
collection of all interpreted rules Cd, where C is from V and d is any ground 
substitution over 2t. 

Definition 1. Let V be a GQLP and 21 G Struct(r). The reduct of V w.r.t. 21, 
denoted red(P,2l), is the set of rules obtained from ground{V ,%) as follows. 

1. Remove every rule r with a literal L in the body sueh that 21 ^ L, where L 

is either negative or a GQ-literal. 

2. Remove all negative literals and GQ-literals from the remaining rules. 

Notice that red{V, 21) is a collection of interpreted Horn clauses; thus, there is a 
least structure 03, denoted 2loo(’^), such that 1 B|to = 21|to (i.e., 05 provides the 
same input to 7^ as 21) and B \= r, for every rule r G red(P,2t). Since red{V,%) 
is an ordinary logic program, the structure 2loo(7^), also called the least model 
of V w.r.t. 21, can be obtained as the least fixpoint of a monotonic operator, cf. 

Eg. 

Definition 2. Let V be a GQLP and let 21q G Struct(ro). An expansion 2t G 
Struct(r) of 21q is a GQ-stable model of V on 2lo, iff it satisfies the fixpoint 
equation 



2l = 2loo(P). 



80 



T. Eiter, G. Gottlob, and H. Veith 



The collection of all stable models ofV is denoted by SM(fP,^o). 

The meaning of V on 21q; denoted A^p(2lo)> is the structure which is the 
intersection of all GQ-stable models ofV on 21q; *-e-; 

■^p(2lo) = f| 

aeSM(-p,ao) 

(If SM(fP,‘Qlo) = 0, then A^p(2lo) is the unique maximal structure 03 such that 
$|ro = %■) 



Example 4- Consider the following program V, which uses the majority quanti- 
fier Qm and the modularity quantifier Q 2 (i.e., the even quantifier). 

Q{x) t- ~'S{x) 

S{x) t- -•Q(x) 

A{x) ^ Qm[Q,S],S{x) 

Q(x) <- Q 2 [A],S{x) 

W{a,b) ^ 

Suppose that tq contains merely the constant symbols a and b, and r contains 
in addition the relation symbols W, A, Q, and S. 

Intuitively, the first two clauses choose complementary extensions for Q and 
S; the third clause assures that S implies A, if Q holds on more individuals than 
S; similarly, the fourth clause assures that S implies Q, if A holds true on an 
even number of elements. 

Consider Herbrand models on r and let ®li = {W{a,b), Q(a), Q(b)}. (We 
use the familiar notation for Herbrand models.) This interpretation is a GQ- 
stable model of P (w.r.t. the unique Herbrand model 9JIo G Struct(To)). Indeed, 
the reduct red(P,DJli) consists of the clauses 

W(a,b)^ A(b) ^ S(b) 

Q(a) <— Q(o-) 'S'(o) 

Q(b) ^ Q(b) ^ S(b) 

A(a) ^ S'(a) 

Clearly, is the least model of this program (w.r.t. 9JIo). Another GQ- 
stable model of P is WI 2 = {W(a,b), B{a,b), S{a), S{b)}. The Herbrand model 
97 I 3 = {W{a, b), B{a, b), Q{a), S'(6)} is not a GQ-stable model of P: red(P, 9 JI 3 ) 
contains the clauses S{b) ^ and Q{b) ^ S{b), which means that the least model 
of red{P, lifts) contains Q{b). In fact, 9Jti and 9 JI 2 are all GQ-stable models of 
P w.r.t. 9Jlo, i.e., SM(P,dJlo) = {9Ki, 9Jl2}- Therefore, 

Let us note some interesting properties of GQ-stable models. The following 
proposition tells that the term “model” is justified. 

Proposition 2. Let Tt be a stable model of a GQLP P. Then, Wt is a model of 
P, i.e., Tl \= y*{P), where V* is the universal closure ofP. 
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Another important property is that GQ-stable models conservatively extend 
the concept of stable models for ordinary logic programs. 

Proposition 3. Let V be a GQLP. If V has no GQ-literals, then dJfl is a GQ- 
stable model ofV ijfdJl is a stable model ofV. 

Therefore, we refer in the sequel to GQ-stable models simply as stable models. 
As Example has shown, a program V may have multiple stable models 
on a structure. Inherent nondeterminism in the choice of the rules enables the 
generation of an enumeration of the universe of a finite structure; we illustrate 
this in the next example. 

Example 5. Gonsider the following program LPsucc, which does not contain ge- 
neralized quantifiers. Therefore, it behaves according to stable semantics for 
ordinary logic programs [21]. We shall refer to this program in Section 

L{x,y) ^L~{x,y) S{x,y) ^ -^S-{x,y) 

L~{x,y) -iL{x,y) S~{x,y) ^ ^S{x,y) 

L{x,z) ^ L{x,y),L{x,z) S~{x,y) ^ L~{x,y) 

T ^ L-{x,y),L~{y,x),x^ y S~{x,z) ^ L{x,y), L{y, z) 

T ^ L{x, x) 

In this program, L and S are binary relations which intuitively describe a linear 
order < on the domain and the associated successor relation, respectively; the 
relations L~ and S~ describe the complements of L and S, respectively. 

The propositional atom T in the heads of the rules is meant to be a logical 
constant for falsity. Such a constant can be easily defined in stable semantics, 
by adding a rule 

r : P± -ir- T, -•Px 

where P± is a fresh propositional letter which does not occur elsewhere. Then, 
in every stable model 9H, proposition T must be false, since otherwise, the rule 
r cannot be satisfied under stable negation. 

It is easy to see that on any finite universe U, the relation L is in every 
stable model a linear order on U, and S is the corresponding successor relation. 
Therefore, we may assume that in any stable model of a program on a finite 
structure, the universe is enumerated. Of course, this does not mean that stable 
model semantics actually defines an order on the domain, because both S and 
L are empty in the intersection of all stable models. 



4 Application: Modular Logic Programming 

In this section, we describe how the formalism of logic programs with generalized 
quantifiers can be fruitfully applied for defining the semantics of logic programs 
which refer to program modules. This approach can be found in mm, and we 
recall the definitions from there. 
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A program module is syntactically described by its interface in terms of 
input and output parameters. Adapted to the logic programming paradigm, a 
logic programming module has an interface 

LP[l- O], 

where the I = /i,... ,Im are predicates which are imported into the module 
(i.e., input predicates) and the O = 0\, . . . , are predicates which are ex- 
ported (i.e., output predicates). If we describe the semantics of a module at the 
extensional level and restrict the number of output relations to n = 1, it is clear 
that such a module is a generalized quantifier. The restriction to a single output 
relation is not essential, however. 

Thus, a module LP\L\ 0\] can be abstractly seen as a GQ (5 lp[I](x), where 
the length of x is the arity of Oi. The semantics of modular logic programs is 
then inherited from the semantics of GQLPs. Notice that this approach is very 
flexible, and allows for easy incorporation of external predicates and functions, 
e.g., averaging and counting predicates. 

4.1 Syntax of Modular Logic Programs 

The syntax of modular logic programs (MLPs) is the one of GQLPs, with the 
difference that the GQ-literals are intended to refer to a logic program, which 
is a logic program module. The similarity type of LP[Q] is the list of arities of 
predicates in Q. 

We shall refer to the calling program as the main program, and the called 
module as the subprogram; the GQ-literals in a MLP are termed call literals, and 
the GQ-predicates call predicates. An atom is a call atom, if its predicate is a 
call predicate. To distinguish ordinary predicates, atoms, and literals from call 
predicates, call atoms, and call literals, we call the former standard predicates 
(atoms, literals respectively.) 

Definition 3 ( [I16| ). A logic program module is a pair p = {LP[Q], P) of a 
module head LP[Q], which has an associated integer n > 0 (the arity), and an 
ordinary logic program P (the body), in which the predicates Q are the input 
predicates and LP is the output predicate having arity n; syntactically, occurren- 
ces of the predicates Q in P are restricted to rule bodies. We require that each 
LP module is uniquely identified by its name LP and the list of the arities of the 
Qi in Q {its similarity type). 

In this definition, the predicates Q can be seen as variables for relations, 
which are substituted at a call by relations over the domain. All relations in a 
module LP[Q] are local; no relations from outside are visible inside a module. 
The similarity type of a module LP[Q] is the input signature Tq of its implemen- 
tation program P. Here, tq is relational, and we do for simplicity not consider 
possible constants and functions in the body P; they could be provided in an 
extended signature tq. 
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Note that the program P does not contain call literals. Hence, the stable 
semantics (as well as other semantics) of ordinary logic programming can be 
readily applied on P. This restriction can be removed and call- resp. GQ-literals 
may be admitted, provided that some basic constraints are obeyed; we shall 
address this issue in Section [T) 

The relations in P are not visible from the main program, and vice versa. 
Thus, relations are imported to the subprogram only if they appear in the scope 
of a call literal, and exported only as the extension of a call literal. 

Definition 4 ([1 fi|). A modular logic program (MLP) is a finite collection LP 
of rules 

H ■<— Li, . . . , Ljn 

where A is a standard atom and the Li are literals, plus a collection C of logic 
program modules such that for each call literal (-i)LP[Q](t) occurring in P, there 
is a module LP[Q'] in C where LP has the arity of t and each Qi € Q has the 
arity of Q\ G Q'. 



4.2 Semantics of Modular Logic Programs 

As discussed above, we can see LP modules as generalized quantifiers. Thus, by 
exploiting this correspondence, we can define the semantics of MLPs in terms of 
the semantics of GQLPs. 

Every LP module fi = {LP[Q],P) can be seen under the semantics 
as a GQ Qc{p) which is associated with the collection C{p) of all structures 
21 = (A, Q',i?'), where the Q' are relations for the predicates Q on A and 
R' = |a|, for any tuple a over A such that P derives the atom LP(a) on 
2lo = (A,Q');i.e., 

dn) = {(A, Q', {a}) I MdiA Q')) h LP{^)} 

We call Qc(p) the module quantifier of fi under M®*. 

Definition 5. Let P be an MLP. The meaning of P under At®* is defined as 
the meaning of P, viewed as GQLP over the collection of GQs associated with 
the LP modules used by P. 



Example 6. Let 971 = {E{1, 2),E{2, 3), E{3, 1), E{2, 4), C/(l)} be a Herbrand mo- 
del of the signature r = {E,U} on the domain {1,2, 3,4}. E^ is the digraph 
depicted below. 

1 3 - 2 



3 



Y 

4 
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Consider the module fj. = {TC[G],Ptc), where Ptc is the program from 
Section |2] The program Ptc has on 2lo = {G(l, 2), G(2, 3), G(3, 1), G(2, 4)} the 
least model DJI' = {G(1,2),G(2,3),G(3, 1),G(2,4)} U {TG(z, j),TG(i,4) | 1 < 
hj < 3}. Thus, a call TC[E] of /r where E is defined in 971, yields that e.g. 
TC[E](1, 1) is true, while TG[i?](4, 1) is false. 

Remark. Certainly, not all generalized quantifiers used have to be program 
quantifiers. In fact, calls to library modules and external modules, formalized as 
generalized quantifiers, can be incorporated easily. 

5 Expressiveness 

Dependency Graph. To obtain a fine-grained analysis of the expressiveness of 
GQLPs, we generalize the familiar concept of dependency graph from ordinary 
LPs to GQLPs. For a GQLP LP, we denote by tlp (or simply r) the signature of 
ordinary predicates of LP, and by r’^p the extension of r with the GQ-predicates. 

Definition 6 ([IS])- Let LP be a GQLP. T/ie dependency graph ofLP, D{LP), 
is the directed graph constructed as follows. The vertices of D{LP) are the pre- 
dicates in rfp; an arc from Pi to P 2 is present in D{LP) if one of 1.-3. applies: 

1. there is a rule Ti(ti) ^ ...,->P 2 {t 2 ), ... in LP. 

2. P\ is a call predicate and P 2 occurs in Q'. 

3. there is a rule Pi{ti) ^ ...,P 2 {t 2 ), ... in LP; 

We color the arc 

red, if 1. applies and both Pi, P 2 are ordinary predicates; 

yellow, if Pi or P 2 is a GQ-predicate; 

green, otherwise (i.e., 3. applies and both Pi and P 2 are ordinary predicates) . 

The color of an arc in D(LP) reflects possible nonmonotonic dependency 
of & Pi on P 2 . Pi. red arc intuitively represents that Pi is nonmonotonically 
dependent on P 2 , i.e., increasing P 2 decreases Pi; a yellow arc represents that Pi 
possibly is nonmonotonically dependent on P 2 (this depends on the properties 
of the respective module involved); finally, a green arc represents that Pi is 
monotonically dependent on P 2 . 

A predicate Pi depends on P 2 , if Pi reaches P 2 in D{LP). If Pi and P 2 
mutually depend on each other, they are called mutually recursive', if neither Pi 
depends on P 2 nor vice versa, they are independent. 

Example 1. Gonsider a GQLP and its dependency graph, (red arcs are symbo- 
lized by label , yellow arcs by “?” ; green arcs are unlabeled) . 



E(x,y) G{x,y,y) 
F{x,y) ^ ~^G{x,x,y) 
Y ^Q^[E,E] 
N 
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According to syntactic conditions on the dependency graph D = D{LP), 
several classes of GQLPs are defined in Table [H 



Table 1. Classes of GQLPs 



Class 


Symbol Description 


unrestricted 


LP arbitrary D 


stratified 


Strat cycles in D have only green arcs 


sermstratified 


SStrat no cycle in D has a red arc 


call recursion free 


CRF no cycle in D has a yellow arc 


call independent 


Cl no path in D has two yellow arcs 


stratified 


SCI cycles in D have only green arcs 


call independent 


and no path in D has two yellow arcs 


recursion-free 


RF D is acyclic 


positive 


Pos D has no red arcs 



Modular Programs. Let Cm be a class of main programs, and let Cs be a class 
of subprograms. Then, Cm{Cs) denotes the class of MLPs with main programs 
from Cm and subprograms from Cs- (Since subprograms do not contain call 
predicates, their dependency graphs do not contain yellow edges.) 

Expressive Power. Let £ be a class of logic programs, and let C be a com- 
plexity class. We say that C has expressivity C (or, equivalently, C captures C) 
over a class of structures 1C, if 

1. for every program LP and structure 21 in /C, Af|fp(2t) is computable in C, 
and 

2. every generic (i.e., isomorphism preserving) mapping / : /C — ^ /C which is 
computable in C is expressible by a program LP under the semantics 

i.e., / = MiV- 

Here, computability of a mapping f in C means that deciding whether a 
relation in /(2l) holds a specific tuple is in C. Semantical restrictions on / 
may be imposed. E.g., monotone co-NP is the class of all monotone maps / 
computable in co-NP. 

The expressive power of ordinary logic programming over finite structures has 
been investigated e.g. in ; see m for a survey. Stable model semantics 

was shown to be quite expressive. 

Proposition 4 (jfiljL LP captures co-NP , or equivalently, all III definable que- 
ries over finite structures. 
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Thus, stable model semantics has the full power of 7T|. In fact, it has been 
shown that even monotone programs can be syntactically captured: 

Theorem 1 (|57|b Monotone NP is captured by the fragment of second order 
logic where all relational symbols (but not the second order relational variables) 
occur only positively. 

5.1 Finite Structures 

In this subsection, we survey the expressiveness of modular logic programs over 
finite structures. The results are compactly summarized in Table E] (LP’^ and 
LP* are defined in Section El. 



Table 2. Expressiveness of MLP over finite structures 



Language 


Expressiveness 


LP(X), CI(X), CRF(X) 
X G {LP,Mon} 
2Pos(LP), SStrat(LP) 


nl 


2Pos(Mon) 


A\ * 


SCI(X), Strat(X), RF(X) 
X G {LP,Mon} 
SStrat(Mon) 


o\ * 


LP, Pos(LP) 


nl 


Pos(Mon) 


monotone n\ 


LP'' 




LP* 


Second-Order Logic 



In this table, an asterisk indicates that the proof requires an enumeration 
of the universe, and thus establishes capturing over ordered finite structures. 
Note, however, that capturing of Ol) over all finite structures is not possible 
unless the exponential Boolean hierarchies EBH and ExpBH collapse to some 
intermediate level; similar results hold for Ab), cf. M- 

Arbitrary Program Modules. The following theorem states the full expres- 
sive power of the formalism. 

Theorem 2. LP(LP) captures II 2 over finite structures. 

Proof. (Sketch) A straightforward algorithm shows that the expressivity of 
LP(LP) is in ilf . 
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For the other direction, consider a formula ^{x\, . . . ,Xn) over signature 
r = (Q). In the language extended by a new relation symbol B of arity n, it 
follows by the extension of Fagin’s Theorem [58] that there exists a formula 
VRBS.ip which is true on a structure 21 iff (p^ C B^. Since stable semantics 
captures co-NP, there exists a program quantifier P 5 for the relation -<3S.(p. 
Then the program 



B(x) <— -'B (x) 

B~ (x) •<— -iB(x) 

P(y) ^ 

R~{y) ^ ^R{y) 

T ^ Ps[Q,B,R] 

A{x) ^ B{x) 

computes precisely the relation A as the intersection of all stable models. □ 

Remark. Note that in LP(LP), the subprograms are arbitrary, so that any 
quantifiers in co-NPcan be computed, and in particular, arbitrary vectorizations 
of a given quantifier. 

The full expressive power of modular logic programs is available in some of 
the syntactical fragments for main programs from above. 

Corollary 1. CRF(LP) and CI(LP) capture TTf finite structures. 

Proof. Follows from the fact that the program in the proof of Theorem is 
call recursion free and call independent. □ 

Corollary 2. SStrat(LP) captures TTf over finite structures. 

Proof. It suffices to show that the complement quantifier (the vectoriza- 
tion of to arbitrary arity k) can be expressed in SStrat(LP): To this end, 
we use additional modules N^, k > 1: 

fV'=[5'] : 

A^'=(x) ^ ^S'(x) 

It appears that the expressive power of the remaining syntactic fragments for 
main programs have expressive power limited to a subclass of on ordered 
structures, this subclass is . For the next results, we make use of the following 
normal form, which has been introduced in and generalized in [ 22 ] . 

Theorem 3 (Stewart Normal Form |5.SI54|22] I . Let f2 be NP-complete un- 
der quantifier-free reductions over ordered finite structures. Then is captured 
over ordered structures by the class of formulas 

3x. I2x.[(^i](v) A -'17 x.[(/32](v) 

where and ip 2 axe quantifier-free formulas. 
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Theorem 4. SCI(LP) captures over all ordered finite structures. 

Proof. (Sketch) Call-Independence means that the oracle calls can be made 
in parallel, and hence Theorem [T] implies that the expressivity is in On 

the other hand, Theorem |3] leads to a program of the form 

^(y ) ^ -Ppi [Q] (x, y ) , ~^Pp 2 [Q] (x, y ) (5) 

□ 

We say that a language C{D) obeys the Stewart Normal Form, if the main 
programs in C{D) can be written in the form d^. Surprisingly, the expressive 
power does not increase if we move from SCI(LP) to Strat(LP). 

Theorem 5. Over ordered finite structures, SCI(LP) = Strat(LP) = RF(LP). 

Proof. (Sketch) Every stratified program P can be partitioned into a finite 
number k of strata ^i, . . . ,Sk- Each stratum Si can be considered as a SCI(LP) 
program over the predicates of the lower strata Sj, j < i, and thus can be 
evaluated by a 6*2 machine Mi. Hence, the evaluation of the program can be done 
by the composition M\ o • • • o Mk of the machines Mi. From the characterization 
of 6*2 in terms of parallel queries (Proposition [I]) it follows that this composed 
computation is in 6*2 • This proves that all mappings in Strat(LP) are in 
Expressiveness for 0 ^ over ordered databases follows from Theorem |H 

The result for RF(LP) follows immediately from the result about Strat(LP) 
and the fact that the program in the proof of Theorem [4| is actually also from 
this class. □ 



Corollary 3. SCI(LP), Strat(LP), and RF(LP) obey the Stewart Normal 
Form. 



Monotone Program Modules. Let us now turn to monotone modules, i.e., 
programs which define a monotone generalized quantifier. Here, we have the 
following. 

Theorem 6 ([17|). Pos(Mon) = Mon, i.e. Pos(Mon) captures the class of 
all monotone coNP queries. 

The intuitive reason for this result is that any program in Pos(Mon) is 
monotone, and its evaluation can be defined in terms of a monotone operator 

ini. 

Let us now proceed to richer fragments of main programs than Pos. For 
stratified program, we obtain the following result. 

Theorem 7. Over ordered finite structures, Strat(Mon), SStrat(Mon), and 
RF(Mon) capture 02 . 
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Proof. (Sketch) With monotone modules, all yellow edges become “mono- 
tone” , and thus all cycles in the given program classes become monotone, and 
therefore by Theorem [6] can be replaced by single calls to a monotone module. 
Collapsing the cycles into single calls we obtain a RF(Mon) program. Since 
RF(LP) is in 02 , membership follows. 

For expressiveness, we use the Stewart Normal Form again, however we are 
now confined to monotone modules. By the following lemma, we can overcome 
this problem by including non-recursive rules which simulate the quantifier-free 
reductions. 

Lemma 1. Over ordered finite structures, there exists a subclass R of (Cl fi 
RF), such that R(Mon) = LP. 

To see the lemma, from Pl57] we know that there exist monotone problems 
which are complete for NP under quantifier-free reductions with successor; such 
reductions can be easily computed by recursion-free programs. 

Thus, by the lemma, it suffices to add a rule like in the proof of Theorem IH 
and we obtain an equivalent program in RF(Mon). □ 

For the remaining fragments of main programs, the expressiveness is higher. 

Corollary 4. LP(Mon), CRF(Mon), and CI(Mon) capture TTf . 

Proof. (Sketch) Membership follows from Theorem O For expressiveness, we 
extend the idea of the proof of Theorem I?] Thus, we employ the program used 
in the proof of Theorem and provide the LP modules by Lemma [2 

Since the proof of Lemma |T] requires ordered domains, we add the rules of 
program LPsucc (cf. Section which in every stable model defines a successor 
relation on the domain. Since the quantifier-free reduction can be simulated 
recursion-free, the result follows. □ 



5.2 Modules under Brave and Skeptical Stable Semantics 

The program classes 2Pos(*) are natural generalizations of the respective Pos(*) 
classes. In the class 2Pos(*), it is possible to call a program module in one of 
two modes, which correspond to the brave and the skeptical variant of the sta- 
ble model semantics. The mode corresponding to the skeptical variant is just 
the one that we used for evaluating modules under the stable semantics so far; 
the mode corresponding to the brave variant follows a dual definition, which 
takes the union of all stable models rather than their intersection. The mode of 
a module call LP[Q](t) is indicated by prefixing the call predicate LP[Q] with 
for the skeptical mode, and with “O” for the brave mode. Similar reasoning 
modes are employed in epistemic logic programming [3]. 

This approach supports programming with modules under different calling 
(or reasoning) modes. 

Clearly, 2Pos(LP) and LP(LP) have the same expressive power since nega- 
tion can be provided by a module (cf. proof of Corollary [21). What happens with 
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2Pos(Mon) ? The brave modality can be seen as a kind of double negation, 
similar to O = -iD-i in modal logic. Therefore, brave calls to a monotone module 
are monotone; this means that 2Pos(Mon) programs amount to positive LPs 
with monotone GQs; for such programs, a least fixpoint computation can be 
applied for evaluating the semantics. The expressiveness must be higher than 
co-NP, since both co-NP-hard and NP-hard modules can be employed. 

To investigate the exact expressibility, we utilize the following strong result. 

Theorem 8 ([55]). Let fl S NP he monotone such that the Hamiltonian Path 
Problem HP is monotone projection-reducible to fl. Then Z\^ is captured over 
ordered structures by least fixed point logic, extended by the family of generalized 
quantifiers for 17. 

Moreover, every Abj property can be defined by a formula 

LFPx.fl ■Gy[</^i](x, max) V -il7z[(^2](x, max) 

where R occurs only positively in the formula, and max denotes a tuple of the 
maximum elements of the order. 



Theorem 9. 2Pos(Mon) captures over ordered finite structures. 

Proof. (Sketch) Since the domain is ordered, we may without loss of generality 
suppose that constants 0, max are available. Membership in Z\^ follows from the 
fact that the modules are monotone by assumption, and therefore, ordinary 
inflationary evaluation of the main program is possible. Since the closure ordinal 
is polynomial in the domain size, a polynomial time algorithm with an NP oracle 
thus obviously exists. 

For expressiveness, we apply Theorem [S| to conclude that there exist two 
modules Phi and Ph 2 which compute Hi and H 2 , under brave and cautious 
semantics, respectively. Now the main program becomes fairly easy: 

i?(x,y) ^ aPHi[Q,R]{x,y) 
i?(x,y) ^ OPH 2 [Q,i?](x,y) 

^(y) ^ -R(max,y) 



□ 

Analogous generalizations 2X(*) can be defined for the other syntactic frag- 
ments X of main programs; however, since they have negation, the brave calling 
mode can be easily emulated by the skeptical mode using auxiliary predicates; 
the expressive power by allowing calls in brave mode thus remains unchanged. 
Therefore, we do not consider such generalizations here. 

5.3 Beyond Finite Structures 

Theorem |2]shows that modular logic programming has the same expressive power 
as disjunctive logic programming over finite structures ng. Thus, it appears that 
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modular logic programming can be translated into disjunctive logic programming 
and vice versa. However, such translations are in general only available for finite 
structures. As shown recently |14| . over infinite Herbrand models (resp. the in- 
tegers) DLP under the stable semantics can express precisely all n\ definable 
relations, and thus has the same expressive power as normal logic programming 
m on these models. 

It is easy to see that the proof of Theorem [2] works over infinite Herbrand 
models resp. the integers as well, and thus establishes that MLP can express 
all II 2 definable relations over these models. Since, as well-known, there are 
definable relations over the integers which are not III definable, it follows that 
MLP is strictly more expressive than DLP on the integers. 

In fact, since the stable semantics expresses all III relations over arbitrary 
structures the proof of Theorem H establishes that modular logic programs 
express over arbitrary structures. Thus, modular logic programming is a 
highly expressive (and complex) formalism in general. 

6 Complexity 

In this section, we survey the complexity of modular logic programs, and address 
the data and the program complexity m of the various classes of modular logic 
programs which we have introduced above. 

Data Complexity. The data complexity of an LP class C is the complexity 
of the mappings Atffp for fixed programs LP G C. All entries in Table El imply 
completeness results for the respective complexity classes, because capturing of 
C by a language C and the existence of a complete problem in C clearly imply 
that £ contains some C-complete problem. Note, however, that the converse 
does not necessarily hold; a language £ can be C-complete while it does not 
capture C. For example, Pos is known to be P-complete with respect to data 
complexity, while it does not capture P (see HD)- Therefore, capturing of C is 
a stronger concept than C-completeness. 

Program Complexity. Besides data complexity, Vardi m introduced the 
concept of program (resp. combined) complexity, program complexitjQ is the 
complexity of evaluating varying logic programs over a fixed input structure 
2to; in combined complexity, both LP and 21 q are varying. Typically, program 
complexity and combined complexity are the same (wrt logspace reducibility) , 
and they are exponentially higher than the data complexity. 

In |23I24| . a method for deriving program complexity results from expressivity 
results has been developed. It is based on complexity theoretic investigations of 
succinct problems, i.e. computational problems, whose instances are not given in 
the usual way as a string, but by a Boolean circuit or by a formula representing 
this string. The representation by a formula is of particular interest, since it can 
be used for simulating succinct input representation within most logics used in 
computer science. 



In |60| . the term expression complexity was used instead of program complexity. Both 
terminologies are widely used. 
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Table 3. Complexity of MLP over finite structures 



Language 


Data Complexity 


Program Complexity 


LP(X), CI(X), CRF(X) 
X £ {LP,Mon} 
2Pos(LP), SStrat(LP) 


m 


EDI = NEXP^P 


2Pos(Mon) 




EA^ = EXP^P 


SCI(X), Strat(X), RF(X) 
X G {LP,Mon} 
SStrat(Mon) 




EO^ = PSPACE^P 


LP, Pos(LP) 


coNP 


coNEXP 


Pos(Mon) 


coNP 


coNEXP 


LP'= 


nl 


EH[ 


LP* 


PH 


EXPSPACE 



The following two syntactic closure properties are easily satisfied by most 
logics: 

Definition 7. A logic C is uniformly interpretation closed, if there exists a 
mapping 

{p, Pi,---, Pr) pI^i/Pi, ---, K/Pr] 

which is computable in logspace and such that for all structures 21, C-expressions 
p, and quantifier-free first order formulas <pi^. . . it holds that 



Definition 8. A logic C is uniformly vector closed, if there exists a mapping 

{if, W) !->• 

which is computable in logspace and such that 

Mod(v3)(l’"l) = Mod((^(l“l)) 

for all binary strings w and C-expressions tp. 

For brevity we say that a logic is uniformly closed if it is both uniformly 
interpretation closed and uniformly vector closed. 

Lemma 2 ( [24] 1 . If C eaptures C and is uniformly closed, then C has program 
eomplexity at least E{C) and Exp{C) under logspace-reductions. 
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Here, E{C) and Exp{C) refer to the exponential analogues of the class C; 
thus E(NF) = NE, and Exp(NF) = NEXPTIME; see j24] for details. 

We apply this lemma for precisely capturing the program complexity of all 
syntactic fragments of modular logic programs that we have considered. The 
results are summarized in Table |3] 

Theorem 10. All program complexity results of Table\^ hold. 

Proof. All languages in this paper are uniformly interpretation closed: For 
a quantifier- free formula ipi, it is easy to write a program simulating (/jj, 
whose output predicate is Ri. Uniform vector closure is obtained by replacing 
all occurrences of variables x by tuples Xi, . . . ,Xn of variables, for n > 1. □ 

In fact, the languages in this paper are even able to simulate succinct circuit 
problems, cf. [24j. Therefore, program complexity can be related to the concept 
of leaf languages [6T162] : it was shown that leaf languages always admit com- 
plete problems under quantifier-free reductions. A straightforward encoding of 
programs as finite structures shows that the above notions of uniformity can be 
restricted to quantifier- free uniformity (see |61|1. and therefore we obtain the 
following corollary: 

Corollary 5. Let C he any LP language considered in this paper. If C captures 
C , then C has program complexity at least Leaf^(C). This holds even with respect 
to quantifier-free reductions. 

Remark. (1) In [41j . a diagonalization argument is given to prove that there are 
arity hierarchies within certain logics. A similar argument shows that the com- 
bined complexity of languages closed under complementation is always higher 
than the expressiveness. 

Indeed, consider programs over strings (i.e., unary alphabets with linear or- 
der). For a program P, let [P] denote its encoding as a string. For a language 
C which is closed under complementation, we define the set AUTSAT{C) = 
{[P] I [P] 1= P}; i.e., AUTSAT{L) contains those programs P which evaluate 
to true on their own representation. The complexity of AUTSAT{C) is clearly 
a lower bound for the combined complexity of C. Moreover, AUTSAT{C) is 
not definable in C: Suppose its complement A{C) is definable in C: Then there 
exists a program P' € C, such that for all programs P £ £, [P] ^ P' iff 
[P] ^ AUTSAT{C). Setting P = P', we arrive at a contradiction. 

As a matter of fact, when we say that the data complexity of a language (e.g. 
NP) is strictly weaker than its program complexity (e.g. NEXPTIME), this is 
just an application of the time hierarchy theorems [32] . which are based on a 
more refined diagonalization technique. 

(2) The same technique can be used to show in an easy way that restricting 
the arity of relation symbols in those classes of programs which are closed under 
complement yield infinite hierarchies wrt to expressiveness: Consider for example 
RF(LP). For A: > 1, RFfc(LPfc) is the class of RF(LP) programs, where all 
relations have maximum arity k. It is easy to see that the combined complexity of 
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RF(LP) remains in , and hence there is a RFfe(LPfe) program LP computing 
Z\(RFfc(LPfc)). Like in (1), we conclude that LP must have arity at least fc+ 1, 
and thus, the result follows. 

Theorem 11. Let C be any logic programming language considered in this pa- 
per. If C captures a complexity class C, which is closed under complement, then 
C has infinite arity hierarchies wrt to expressiveness. 

7 Nested Program Modules 

It is evident that a modular logic program can again be used as a subprogram to 
a new logic program; proceeding this way, we obtain nested modular programs 
of arbitrary depth. A program which is built up this way and which does not 
contain cyclic dependencies of subprograms (i.e., no recursive calls of modules), 
is called hierarchical. 

For finitely nested, hierarchical programs, a semantics can be inductively de- 
fined, based on the semantics of programs of nesting depth 2 which we presented. 
In particular, stable models of such programs can be naturally defined, and stra- 
tified, positive, etc. fragments of programs can be considered. We do not carry 
out this here; the formalization should be clear. 

We extend out notation for nested classes in the following way. For any fc > 1, 
LP'^ is the class of all modular logic programs with nesting depth bounded by 
fc, and LP* = Ufc>i LP''- In particular, LP = LP^ and LP'^+i = LP(LP'=). 

Theorem 12. LP*^ captures the class of PH, and LP* captures PH. 

Proof. Analogous to Theorem [2 □ 

Thus, modular logic programming with nestings is as expressive as full second- 
order logic over finite structures (cf. |58]). In fact, this is an interesting difference 
to extensions of first order logic by generalized quantifiers: Although logic pro- 
grams can be considered as a nested quantifier, there is no collapse to the 
level, as there is in extensions of first order logic by NP quantifiers |^; the 
reason is that in stable model semantics, there is a direct access to relational 
variables which is not the case for generalized quantifiers. 

Moreover, from the considerations in Section Fb., SI we can conclude that nested 
modular logic programs allow to express full second order logic over arbitrary 
structures. Thus, LP* programming with nestings is a tremendously expressive 
language. 

An inductive definition of different fragments by nesting programs from dif- 
ferent fragments of LP is straightforward. For example, we can define classes 
such as Strat(LP(Strat)), Strat(CRF(LP)) etc. 

The capturing results from Table [2 can be lifted to all constituent classes of 
PH by appropriate classes of nested MLPs. For example, Strat(SStrat(LP)) 
captures 6*3, and it obeys the Stewart Normal Form in the main programs. Note 
that LP(Strat(LP)) collapses to LP(LP) since the logspace computations in 
the medium module can be transferred to the main program. 
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8 Conclusion 

The extension of logic programs by generalized quantifiers is a useful tool for a 
natural formalization of modular logic programs. 

As shown above, the formalism for modular logic programming described 
in [T7ITHI has high expressive capability and equals the power of second order 
logic. Over finite structures, it captures the polynomial hierarchy, and natural 
fragments of the language capture the constituent classes in this hierarchy. 

Several issues remain for further research. On the semantical side, an in- 
triguing issue is a semantics for nested modular logic programs which are not 
hierarchical, i.e., modular logic programs which use recursive calls of modules. 
Furthermore, it would be interesting to experience semantics for modular logic 
programs which are based on semantics different from the stable model seman- 
tics; in particular, for semantics which use multi-valued logics (see m for several 
alternative semantics of logic programs). 

Another direction of research is the study of the effect of restricting arities of 
generalized quantifiers on expressiveness. In particular, it would be interesting 
to see arity hierarchies for program classes capturing complexity classes which 
are not closed under complementation (e.g., LP(LP)). Moreover, the picture of 
the arity hierarchies in the other classes of programs remains to be refined. 
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Abstract. A notion of generalized quantifier in computational comple- 
xity theory is explored and used to give a unified treatment of leaf lan- 
guage definability, oracle separations, type 2 operators, and circuits with 
monoidal gates. Relations to Lindstrom quantifiers are pointed out. 



1 Introduction 



In this paper we develop a unified view at some notions that appeared in compu- 
tational complexity theory in the past few years. This will be in the form of opera- 
tors transforming complexity classes into complexity classes. Each such operator 
is given in the form of a quantifier on strings. This will immediately subsume as 
special cases the well known universal, existential, and counting quantifiers ex- 
amined in various complexity theoretic settings |SM73IWra77|Wag86b|Wag86a| 
| |Tor91| . But also a lot of constructions from other subareas of complexity theory 
can best be understood in terms of such operators. These include circuits with 
arbitrary monoidal gates | |BIS90IBI94| . oracle operators |BW96IBVW96| . leaf 
languages (introduced in |BCS92|Ver93l and examined for different computa- 
tion models in |HLS~*~93IJMT94|CMTV96n . We survey some results from these 
areas and establish some new connections. 

In finite model theory, examinations of the expressive power of various logics 
enhanced by Lindstrom quantifiers form a very well established field of active 
research. Descriptive complexity theory has characterized a great bulk of com- 
plexity classes by such logics. We will show that classes defined by our general 
operator can in a uniform way be characterized by model theoretic means using 
Lindstrom quantifiers. 

In the following, we assume some familiarity of the reader with basic formal 
language theory (refer to [R.S97] L basic complexity classes and resource-bounded 
reducibilities (refer to the standard literature, e.g. Pap94IBC94IBDG95J : all 
complexity classes that appear in this paper without definition are defined in 
[l,Toh90| L as well as with the basics of finite model theory (refer to |Vaa94IEF95j L 



2 Definition 

Given a language A over some alphabet U, we denote the characteristic function 
of A by XA, be. for all x G U*, xa(x) = 1 ii x € A, and xa{x) = 0 otherwise. 

J. Vaananen (Ed.): ESSLLP97, LNCS 1754, pp. 99 41231 1999. 

(c) Springer- Verlag Berlin Heidelberg 1999 
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We will always assume some order on the alphabets we use; therefore it 
makes sense to talk about the lexicographic order ^ of if*, and for x,y G if*, 
X ^ y, we define the characteristic string of A from x to y as \A [x ■ ■ - y] =def 
Xa(x)xa(x + 1) • • • XA{y)- Here, x + 1 denotes the successor of x. In fact, we 
will presuppose an underlying bijection between if* and the set N of natural 
numbers, and we use the notation XA[i ■ • • j] for i,j G N. 

Let (•, •) denote a standard pairing function. For a set H C if* and a string 
X G if*, define =def {y \ {x,y) € A}. 

Looking now at the well-known characterization of the polynomial hierar- 
chy by polynomially length-bounded universal and existential quantifiers 3^, 
|WFi77] . the following is clear: 

— A language L is in NP if and only if there is a language A G P and a function 
/ computable in polynomial time such that for all x, 

x£L <;=^ [0 . . . /(a:)] G (0 -I- 1)*1(0 -I- 1)* (i.e., contains a “1”). 

— A language L is in coNP if and only if there is a language A G P and a 
polynomial p such that for all x, 

X € L XA*[0 . . . f(x)] G 1* (i.e., consists out of “l”s only). 

An analogous result holds for the class PP, which was characterized in 
||Wag86a| in terms of the so called polynomially length-bounded counting quan- 
tifier C^: 

— A language L is in PP if and only if there is a language A G P and a function 
/ computable in polynomial time such that for all x, 

X £ L XAa,[Q ■ ■ ■ f{x)] contains more “l”s than “0”s. 

The class US (for unique solution) is defined by polynomial time nondetermi- 
nistic Turing machines M which accept an input x if and only if there is exactly 
one accepting path in the computation tree of M on x. 

— A language L is in US if and only if there is a language A G P and a function 
/ computable in polynomial time such that for all x, 

x£L ^ XaJ0.../(x)]g0*10*. 

Thus we see that here the semantics of quantifiers is defined by giving lan- 
guages over the binary alphabet {E =def (0 3- 1)*1(0 -I- 1)* for 3^*, U =def 1* for 
\/P, and maj =def { w G {0, 1}* | w contains more “l”s than “0”s } for C^). The 
following generalization now is immediate: 

Let B G {0, 1}*, let /C be a class of sets, and T he a class of functions from 
S* -A N. Define the class {B)^K. to consists of all sets L for which there exist 
some A £ K. and some function f £ E such that for all a; G A*, a; G L 
XaJO . . . /(a;)] G B. 
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We use the following shorthands: Write (A)p/C ((A)'°s/C, (A)p^°s/C, resp.), if 
T is the class of all functions from if* — >■ N computable in polynomial time 
(logarithmic time, polylogarithmic time, resp.) on deterministic Turing machi- 
nes (i.e., T = T = FDLOGTIME, T = FPOLYLOGTIME, resp.). For 
sub-linear time bounds we use Turing machines with index tape and random 
access to their input, working in the unrestricted mode (for background, re- 
fer to | CC95IRV97p . Observe that a function / S FP is polynomially length- 
bounded, / € FDLOGTIME is length-bounded by some function c • logn, and 
/ € FPOLYLOGTIME is polylogarithmically length-bounded. If £ is a class of 
languages, then (£)-^/C =def 

If we take the above three languages E, U, and maj, and look at different 
function classes T, we get the existential, universal, and counting quantifier for 
various length-bounds. 

The above definition appeared in jVo19fibj and (for the special case T = FP) 
in [BS^ . 

3 Polynomial Time Leaf Languages 

The most examined special case of our general operator is probably the poly- 
nomial time case, i.e. the base class /C is the class P (and T = FP). In this 
case there is a very intuitive way of visualizing the operator via so called Zea/ 
languages. 



3.1 Definition 

In the leaf language approach to the characterization of complexity classes, the 
acceptance of a word input to a nondeterministic machine depends only on the 
values printed at the leaves of the computation tree. To be more precise, let M 
be a nondeterministic Turing machine, halting on every path printing a symbol 
from an alphabet Y, with some order on the nondeterministic choices. Then, 
leafstring^(a;) is the concatenation of the symbols printed at the leaves of the 
computation tree of M on input x (according to the order of M’s paths given 
by the order of M’s choices). Given now a language B C {0,1}*, we define 
Leaf'^(i?) = { a; I leafstring'^(a::) G S }. 

Gall a computation tree of a machine M balanced, if all of its computation 
paths have the same length, and moreover, if we identify every path with the 
string over {0, 1} describing the sequence of nondeterministic choices on this 
path, then there is some string 2 such that all paths y with |?/| = \z\ and y z 
(in lexicographic ordering) exist, but no path y with y >- z exists. 

A leaf language BCE* now defines the class BLeaf^(S) of all languages 
L for which there exists a nondeterministic polynomial time machine M whose 
computation tree is always balanced, such that L = Leaf^(B). Let C be a class 
of languages. The class BLeaf^(C) consists of the union over all B G C of the 
classes BLeaf^(i?). 
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This computation model was introduced by Bovet, Crescenzi, and Silvestri, 
and independently Vereshchagin [BCS92IVer93] and later examined by Her- 
trampf, Lautemann, Schwentick, Vollmer, and Wagner [HLS^93] . and Jenner, 
McKenzie, and Therien | JMT94| . among others. See also the textbook |Pap94 
pp. 504f]. 

Jenner, McKenzie, and Therien also considered the case where the computa- 
tion trees are not required to be balanced. For that case, let B be any language. 
Then, the class Leaf^(i?) consists of those languages L for which there exists a 
nondeterministic polynomial time machine M without further restriction, such 
that L = Leaf^(B). Let C be a class of languages. The class Leaf^(C) con- 
sists of the union over all B G C of the classes Leaf^(iJ). (Strictly speaking, 
the definition of balanced given in | JMT94| is different from ours and, at first 
sight, slightly more general. However, it is easy to see that both definitions are 
equivalent.) 

The reader now might wonder about the seemingly unnatural condition that 
the nondeterministic choices of M are ordered. In fact, most complexity classes 
of current focus can be defined without this assumption — in this case the leaf 
language B has the special property that we can permute the letters in a given 
word without affecting membership in B. (Cf. our results on cardinal langua- 
ges in Sect. 13.31 below.) However, strange classes where the order of the paths 
is important for their definition are conceivable, and the results presented be- 
low, especially the oracle separation criterion (Theorem |6]) , also hold for these 
pathologic cases. 

The following connection to our generalized quantifier now is not too hard 
to see. 

Theorem 1. Let B C {0, 1}*. Then (H)PP = BLeaf^(H). 

Proof sketch. (C) Let L G (B)PP, x G L [0 . . . /(x)] G B. The non- 

deterministic machine, given x, branches on all possible second inputs y in the 
range 0, . . . , /(x), and outputs xa{x, y). 

(3) Let L G BLeaf^(B) via the nondeterministic machine M. Computation 
paths of a nondeterministic machines can be followed in polynomial time if the 
nondeterministic choices are known. Defining A to consist of all pairs (x,p) such 
that p is a sequence of nondeterministic choices leading to a path of M that 
outputs “1” and /(x) to be the number of paths of M on input x, we have 
xG L XA, [0 . . . /(x)] G B. □ 

The definition of leaf languages allows for languages B not necessarily over 
the binary alphabet. If we want to come up with a connection to our generalized 
quantifier also for such B, we face a problem. In the definition in Sect. |2]the 
binary alphabet seems essential. Fortunately, for every B there is usually a B' C 
{0,1}* such that BLeaf^(i?) = BLeaf^(iJ'), where B and B' are of the same 
complexity. In most cases, B' can simply be obtained from B by block encoding 
(then B and B' are FO-equivalent). We come back to this point in the next 
subsection. 
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3.2 The Complexity of a Leaf Language 

In [HLS+931 the question how complex a leaf language must be in order to 
characterize some given complexity class K was addressed. Let us start by con- 
sidering some examples. 

At great number of classes can be defined by regular leaf languages. This is 
obvious for NP, coNP and US as we saw in the previous section, for Mod^P (all 
words with a number of “l”s divisible by fc), but also true for higher levels of the 
polynomial hierarchy (see below) and the boolean hierarchy over NP (e.g. the 
class NP A coNP can be defined via the set of all words such that the string 
“010” appears at least once, but the string “0110” does not appear). 

For other complexity classes, context-free languages come immediately to 
mind. PP can obviously be defined by the language maj from the previous sec- 
tion. Recalling the characterization of PSPACE via polynomial time alternating 
Turing machines, it is clear that the set of all (suitably encoded) boolean ex- 
pressions involving the constants “true” and “false” and the connectives AND 
and OR that evaluate to “true” , is an appropriate leaf language. 

The question however arises if we can do better here. It was shown in 
pdLS+98j . that in the case of PSPACE there is a regular leaf language. 

Let S'5 denote the word problem for the group of permutations on five ele- 
ments (suitably encoded over the binary alphabet), i.e. S'5 consists of sequences 
of permutations which multiply out to the empty permutation. 

Theorem 2. (Ss)PP = PSPACE. 

Proof sketch. For the inclusion from left to right, just observe that a PSPACE 
machine can traverse the whole computation tree of a given nondeterministic 
machine to evaluate the product over S5. This simulation then stops accepting 
if and only if the result is the identity permutation. 

For the other direction, we are given a language L G PSPACE. Then there 
is a polynomial time alternating Turing machine accepting L. Thus, for every 
input w, machine M defines a polynomial depth computation tree T{w) where 
the leafs carry values 0 or 1 and in the inner nodes the functions AND and 
OR are evaluated, w G L the root of this tree evaluates to 1. As a first 
step we transform this tree into a tree T'{w) where in all the inner nodes the 
function NOR is evaluated. This can easily be achieved since the NOR function 
constitutes a complete basis for the boolean functions. 

As a second step we now “simulate” NOR in S^. This simulation is essentially 
due to David Barrington [Ba,r89] . Let b,c^d,e^f be the following permutations 
from the group S 5 : 

b = (23)(45), c = (12435), d = (243), e = (345), / = (152) 

Further let Og be the empty permutation, denoted by Og = (), and let oi = 
(12345). Now consider the following product in S'5 including the variables x and 

y- 



w{x,y) = aobx'^cy'^dxeyf 



104 



H. Vollmer 



Simple calculations show that w(ao,ao) = a\ and w(ao,ai) = u>(ai,ao) = 
w{ai,ai) = tto- Thus coding the value true by oi, and false by oq, we can 
view w as the NOR-operation applied to x and y. 

Now replace every appearance of a “NOR”-node in T'{x) with sons x and y 
by a binary subtree of height 4 whose 16 leaves are 

aobxxxxcyyyydxeyf 

Thus we accept the input w if and only if the leaf string evaluates to ai. 
Taking B to be the regular language 

B =def { a; I a; is a string of elements from which evaluates to Ui } , 

we then get Leaf^(i?) = PSPACE. It is easy to go from B to the word problem 
S's (by just adding one more factor and since we have an identity element 

which we can insert arbitrarily in the leaf string to fill gaps in the computation 
tree in order to make it balanced, we get PSPACE = Leaf^(R) = BLeaf^(S' 5 ) = 
(^5)PP. □ 

The question now of course is what is so special about the language S' 5 . 
What can be said more generally? Using deep algebraic properties of regular 
languages exhibited in | The81IBT88| (see also the textbook |Str94| l one can 
show the following. 

Let PH denote the union of all classes of the polynomial hierarchy ISM73I , 

i.e. PH = NPUNP^^UNP^^ U- • • . Let MOD-PH denote the oracle hierarchy 
constructed similarly, but now allowing as building blocks not only NP but also 
all classes Mod^P for arbitrary A: £ N. 

Theorem 3 ( |HLS+93p . 

1. Let A he a regular language whose syntactic monoid is non-solvable. Then 
(A)PP = PSPACE. 

2. Let SOLVABLE denote the class of all regular languages whose syntactic 
monoid is solvable. Then (SOLVABLE)PP = MOD-PH. 

3. Let APERIODIC denote the class of all regular languages whose syntactic 
monoid is aperiodic. Then (APERIODIC)PP = PH. 

Regular leaf languages for individual levels of the polynomial hierarchy can 
also be given. For example Aj can be defined over S = {a, 6 , c} by A*ca+cA*, 
intuitively: “there is a block consisting out of ‘a’s only” . This is an 3V predicate 
directly reflecting the nature of A 2 -computations. If we now chose a simple block 
encoding this might lead us out of the aperiodic languages. However, we may 
proceed as follows: Define A 2 = (0 -I- 1)*11(010)~''11(0 -I- 1)*. It is clear that this 
leaf language defines a subclass of A| — just check that there are two substring 11 
such that in between we have a sequence of occurrences of the 3-letter string 010; 
this is an 3V condition. On the other hand, suppose we are given a A 2 machine 
M, i.e. an alternating machine with computation trees consisting of one level of 
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3 nodes followed by a second level of V nodes; i.e. the initial configuration is the 
root of an existential tree where in the leaves we append universal subtrees. We 
transform this into a tree where we use the substring “ 11 ” in the leafstring as 
separator between different V subtrees, and within each such subtree we simulate 
an accepting path by the 3 leaf symbols “010” and a rejecting path by the symbol 
“0” . Then M produces a tree with at least one universal subtree consisting out of 
only accepting paths iff the leaf word of this simulation is in A 2 . can similarly 
be defined via A 3 = (0 + l)*lllA 2 lll (0 + 1)*. This generalizes to higher levels 
of the polynomial hierarchy. With some care one can show that A 2 and A 3 are 
in levels B 2 and B 3 , resp., of the Brzozowski-hierarchy of regular languages. This 
hierarchy of star-free regular languages measures the nesting depth of the dot 
(i.e. concatenation) operation. For a formal definition see |Fi17fi] . More generally 
the following holds: 

Theorem 4 f |HLS+fl.3] T (yBfc)PP is the boolean closure of the class 

Let us now come back to the question if PP (for which we gave a context-free 
leaf language above) can also be done by a regular language. 

Corollary 1. PP is not definable via a regular leaf language unless either W = 
PSPACE or PP C MOD-PH. 



Proof. If there is a regular leaf language L for PSPACE, then there are two cases 
to consider: either L is non-solvable (in this case PP = PSPACE) or L is solvable 
(then PP C MOD-PH). 



In |HLS~*~93 leaf languages defined by restricting resource bounds as time and 
space were examined. It was shown that the complexity class obtained in this way 
is defined via the same resource, but the bound is one exponential level higher, 
for example (P)PP = EXPTIME, (NP)PP = NEXPTIME, (LOGSPACE)PP = 
PSPACE, (PSPACE)PP = EXPSPACE, and so on. Denoting the levels of the 
alternating log-time hierarchy |Sip83| by Xjf® (fc G N), we get the following 
special case: 



Theorem 5. (X[°®)PP = LfP. 



3.3 Some Complexity Theoretic Applications 

Normal Forms The characterization of PSPACE (Theorem was somewhat 
surprising, since it points out a very restricted normal form for PSPACE compu- 
tations. Cai and Furst defined a class /C to be K,' -serializable, if every /C compu- 
tation can be organized into a number of local computations ci , . . . , Cr (which in 
turn are restricted to be K,' computations), each passing only a constant number 
k of bits as the result of its computation to the next local computation. The 
sequence ci , . . . , is uniform in the sense that there is one K,' program that 
gets as input only the original input, a number i, and a string of k bits, and 
computes the A:-bit-result of cfs computation. Please refer to jCFQlj for a for- 
mal definition. Machines as just described are also called bottleneck machines. 
The bottleneck refers to the restricted way of passing information onwards. 
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Corollary 2 ( |HLS+93| 1. PSPACE is -serializable. 

Proof sketch. Let L S BLeaf^(S' 5 ) via machine M. The information passed from 
one computation to the next will be an encoding of an element of the group S^. 
Each local computation uses its number to recover from it a path of the nondeter- 
ministic Turing machine. (If the number does not encode a correct computation 
path, then we simply pass the information we get from our left neighbor onwards 
to the right.) The leaf symbol on this path is then multiplied to the permutation 
we got from the left, and the result is passed on to the right. This can be done 
in AC° since computation paths of polynomial time Turing machines can be 
checked in AC°. (A computation path consists not only out of M’s nondetermi- 
nistic choices, but is a complete sequence of configurations of M.) 

□ 

The power of bottleneck machines was examined in detail in |Her97j . He gave 
a connection between these machines and leaf languages defined via transforma- 
tion monoids. The power of bottleneck machines as a function of the number of 
bits passed from one local computation to the next was determined. 



Oracle Separations The original motivation for the introduction of leaf langu- 
ages in [BCS92IVer98J was the wish to have a uniform oracle separation theorem. 
Usually when relativized complexity classes are separated, this is achieved by 
constructing a suitable oracle by diagonalization, usually a stage construction. 
Bovet, Crescenzi, Silvestri, and Vereshchagin wanted to identify the common part 
of all these constructions in a unifying theorem, such that for future separations, 
one could concentrate more on the combinatorial questions which are often diffi- 
cult enough. They showed that to separate two classes defined by leaf languages, 
it is sufficient to establish a certain non-reducibility between the defining langua- 
ges. Let A, B C {0,1}*. Say that A is poly logarithmic time bit-reducible to B, in 
symbols: A B, if there are two functions /, g computable in polylogarithmic 
time such that for all x, x G A f{x, 0)f{x, 1) • • • f{x, g{x)) G B. 

Theorem 6 ( [BCS92|Ver93| b LetA,B C {0,1}*. Then A B if and only 
if for all oracles Y , the inclusion (A)PP^ C (H)PP^ holds. 

Observe that A B is just another formulation for the containment of A 
in (H)P'°sPOLYLOGTIME, which in turn is equivalent to the inclusion of the 
class (A)Pi°«POLYLOGTIME in (H)P'°sPOLYLOGTIME. 

Corollary 3. Let A,BC {0,1}*. Then we have: 

(A)P'°spOLYLOGTIME C (B)P^°spOLYLOGTIME 

if and only if for all oracles Y , the inclusion 



holds. 



(A)PP^ C (B)PP^ 
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In |BS97j . Theorem[6]was strengthened as follows: It was shown that (A)p/C C 
(i?)P/C for all nontrivial classes /C if and only if A is reducible to by monotone 
polylogarithmic-time uniform projection reducibility. Refer to their paper for 
details. 

Observe that a polylogarithmic time bit-reduction cannot (simply because of 
its time bound) read all of its input. This often allows one to prove A B by 
an adversary arguments. We give a very simple example. 

Example 1. Let £1 = (0 -h 1)*1(0 -h 1)*, C7 = 1* as in Sect. |2] Then (£;)PP = NP 
and (C/)PP = coNP. Suppose U E. The input a: = 1" must be mapped by 
this reduction to a word with at least one “1” . The computation leading to this 
“1” however cannot read all of x. If we now define x' by complementing in a: a 
bit which is not queried, then again x' will be mapped to a string in E, which 
is a contradiction. Thus U E, and hence there is an oracle separating coNP 
from NP. 

Vereshchagin in [Ver98] used Theorem El to establish all relativizable inclu- 
sions between a number of prominent classes within PSPACE. His list contains 
besides the classes of the polynomial time hierarchy also UP, FewP, RP, BPP, 
AM, MA, PP, IP, and others. 

A very satisfactory application of Theorem E] was possible in the following 
special case. Say that L C E* is a cardinal language, if membership in L only 
depends on the frequency with which the elements of E appear in words. This 
means that if A = {a\, . . . ,Ofc} we can associate L with a set N{L) C N^, in 
such a way that G L iff there is a (ui, . . . ,Vk) G N{L) where occurs in w 
exactly Vi times {1 < i < k). {N{L) is the image of L under the Parikh mapping: 
N{L) = EsiL).) Say that L is of bounded significance if there is a number 
m G N such that for all (ui , . . . ,Vk) we have 

{vi, . . . ,Vk) € N{L) (min(ui,77i), . . . ,min(tife,m)) e A^(L). 

Using Ramsey theory, Hertrampf in |Her95a| proved the following: 

Theorem 7 ( |Her95al ) . There is an algorithm that, given two cardinal langu- 
ages A,B of bounded significance, decides if (A)PP^ C (i?)PP^ for all oracles 

Y. 



Pushing his ideas just a bit further, the following was proved: We say that 
p: — >■ N is a positive linear combination of multinomial coefficients if p{v) = 

some 2 G N^', G N (for u < z, the order taken component- 
wise). 

Theorem 8 ( |CHVW97] L Let A, B he cardinal languages of bounded signifi- 
cance over a k element alphabet. Then A B if and only if there are fun- 
ctions pi, . . . ,pk'. N which are positive linear combinations of multino- 

mial coefficients, such that for all v = {v\, . . . ,Vk), v G N(A) if and only if 
{pi{v),...,pk{v)) G N{B). 
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In other words, if such k functions do not exist, then there is an oracle 
separating (A)PP from (i?)PP. Thus we see that the oracle separation criterion 
Theorem [6] leads to a very strong statement in the context of cardinal languages. 
This result was used in |CHVW97| to establish a complete list of all relativizable 
inclusions between classes of the boolean hierarchy over NP and other classes 
defined by cardinal languages of bounded significance. 

Valiant’s counting class #P is of course strongly related to the notion of 
cardinal languages. In the case of #P we just deal with the binary alphabet, 
and we count the number of “l”s in a leaf string. Closure properties of #P, 
that is operations that don’t lead us out of the class, play an important role 
to establish inclusions between complexity classes; e.g. Toda’s result PH C P^^ 
[rPodQlJ and Beigel, Reingold, and Spielman’s proof that PP is closed under 
intersection [HP.SQlj both heavily build on the fact that #P is closed under 
certain sums, products, and choose operations. 

Similar to Theorem [HI one can obtain the following: 

Theorem 9 f |HVW95] b A function / : N is a relativizable clo- 

sure property of ffP (i.e., relative to all oracles, if h\, . . . ,hu G #P then also 
f{hi , . . . , hk) S #P ), if and only if f is a positive linear combinations of multi- 
nomial coefficients. 



Circuit Lower Bounds Circuit classes as leaf languages have been considered 
in |CMT V 96IVol96a] . For background on circuit complexity, we refer the reader 
to |Str94| . It is immediate from Theorem |2] that (NC^)PP = PSPACE. Additio- 
nally one can prove e.g. that (AC°)PP = PH, and that (TC°)PP is the counting 
hierarchy CH, defined in | Wag86b|Wag86a| as PP U PP^^ U PP^^ U • • • . Finer 
results are given in [Vol97f 

Building on leaf language characterizations, the circuit class TC° (where 
we require logtime uniformity) was separated from the counting hierarchy in 
[ICMT V 96] . This was improved by Allender [A1196j to the following separation. 



Theorem 10. TC° yf PP. 



Proof sketch. We sketch the proof of the weaker result from jCMTV9~6] . Suppose 
that TC° = CH. Then we have TC° = CH = BLeaf^(TC°) = BLeaf^(CH) A 
EXPTIME, thus P A EXPTIME, which is a contradiction. Allender now obser- 
ved that this can be extended to show that any language complete for PP under 
TC° reductions cannot be in TC'^. □ 



In the non-uniform case no similar lower bound for TC*^ is known. If we relax 
the uniformity condition just a little bit, we know that 

(logspace-uniform AC°)PP = PSPACE 

(thus also BLeaf^ (logspace-uniform TC°) = PSPACE). This shows that logtime- 
uniformity is critical in the above proof. 
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In Corollary |3] it became clear that the oracle separability of two polynomial 
time classes is equivalent to the absolute separability of two lower classes with 
the same acceptance paradigm. A similar relation is known between polynomial 
time and constant depth circuit classes. E.g. building on previous work by Furst, 
Saxe, and Sipser |FSS84| . Yao in his famous paper used a lower bound for the 
parity function to construct an oracle separating PSPACE from the polynomial 
hierarchy |Yao85| . This connection has been exploited a number of times since 
then. 

The formal connection between Theorem El and the Furst, Saxe, Sipser ap- 
proach to oracle construction has been given in |Vo197j . The main observation 
that has to be made is that <^*-i-eductions can be performed by (uniform) 
qAC° circuits. qAC° stands for quasipolynomial AC° |Pa,r92] . i.e. unbounded 
fan-in circuits of constant depth and size (Similarly we will also use 

qTcO for quasipolynomial size TC'^ circuits, and qNC^ for quasipolynomial size 
NC^ circuits.) 

Theorem 11. Let A,B {0,1}*. Then we have: A ^ (il)P*°®qAC° if and only 
if (A)P*°SqAC° 2 (B)P^°®qAC'^ if and only if there is an oracle Y such that 
(A)PPH^ g (B)PPH. 

This theorem can be used to attack the “nagging question” [l’or97] how to 
separate superclasses of P^^ from PSPACE. Some special cases are the following. 

Corollary 4. S 5 ^ qTC'^ if and only j/qTC° yf qNC^ if and only if there is an 
oracle separating the counting hierarchy from PSPACE. 

Proof sketch. Under the assumption S 5 G qTC'^, the following inclusion chain 
holds relativizably: 

PSPACE = BLeaf^( 55 ) C BLeaf^(qTC°) = CH. 

This proves the direction from right to left. For the other direction, if relative to 
all oracles PSPACE C CH then S 5 polylogarithmic time bit-reduces to qTC°, 
but this class is even closed under qAC° reductions. □ 

Define par =def { w G {0, 1}* | the number of “l”s in w is odd }, and let 
maj be as in Sect. E] 

Corollary 5. S 5 ^ (maj)P^°®(par)P'°SqAC° if and only if there is an oracle 
separating pp®^ from PSPACE. 

Proof sketch. If PSPACE C pp®^ then polylogarithmic time bit-reduces to 
a language in the class (maj)P'°®(par)P^°SqAC°, and therefore is even in this 
class (it is closed under <l^). 

On the other hand, if S '5 G (maj)P'°s(par)P'° 8 qAC°, then PSPACE = 
BLeaf^(S' 5 ) C BLeaf^((maj)P'°s(par)P'°SqAC°) = PP®p'’“ = PP®^. □ 

A refinement of Theorem 1111 and further investigations along these lines can 
be found in |M97]- 
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3.4 Definability vs. Tree Shapes 

Our quantifier from Sect. [21 coincides as we saw in the polynomial time context 
with leaf languages for balanced computation trees. The unbalanced case has 
also attracted some attention in the literature. It was observed in jHVWQfi] 
that the relativization result from [BCS92IVer93J does not hold in the case of 
unbalanced trees. Thus, part of the motivation to consider this construct is 
gone. Nevertheless definability questions are also interesting in this case. The 
just mentioned observation even makes a systematic comparison of both models 
a worthwhile study. 



Balanced vs. Unbalanced Trees In [HVW96] the question of definability of 
the polynomial hierarchy was addressed. As mentioned earlier in Theorem E] 
the classes of the log-time hierarchy exactly define the classes of the polynomial 
hierarchy. However, in the case of unbalanced trees, one can somehow use the 
tree structure to hide an oracle that is able to count paths. More formally. 

Theorem 12 ( HVW96n . LeafP(4°s) = 



The Acceptance Power of Different Tree Shapes Hertrampf [Her 9 5b] 
considered besides the above two models also the definition of classes via leaf 
languages for computation trees which are full binary trees. The obtained clas- 
ses are noted by FBTLeaf^(-). Though trivially for every B C {0, 1}* we have 
FBTLeaf^(i?) C BLeaf^(B) C Leaf^(i?), Hertrampf proved the somewhat 
counterintuitive result, that the definability power by arbitrary single regular 
languages does not decrease but possibly increases as the tree shapes get more 
and more regular; that is for every regular language B there is a regular langu- 
age B' such that Leaf^(il) = Leaf^(iJ') = BLeaf^(i?'), and for every regular 
language B there is a regular language B' such that BLeaf^(H) = BLeaf^(H') = 
FBTLeaf^(H'). 



Definability Gaps In the case of arbitrary tree shapes, Borchert et al. were 
able to prove the existence of definability gaps. In particular, the following was 
shown. 

Theorem 13. Suppose the polynomial hierarchy does not collapse, and let B be 
an arbitrary regular language. 

1- IfP C Leaf^(il) C jC, then Leaf^(il) = P or Leaf^(H) = 1C, where K. is one 
of the classes NP, coNP, or ModpP (for some prime number p) [Bor94| . 

2. If NP C Leaf^(H) C coUS, then Leaf^(H) = NP or Leaf^(H) = coUS 
[BKS96] (analogously for coNP and US/ 



We come back to questions of this kind in Sect.[B] 
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4 Other Resource Bounds 

4.1 Circuit Classes 

Corollary |2] easily yields the following: 



Corollary 6. (S' 5 )pAC° = PSPACE. 

This coincidence between (-)PP and (•)pAC° holds under more general cir- 
cumstances. Let J\f denote the set of all languages L C E* that contain a neutral 
letter e, i.e. for all u,v € E*, we have uv G L uev G L. 

Theorem 14. If B G N then {B)PP = {B)pAC°. 



Proof sketch. Computation paths of nondeterministic Turing machines can be 
checked in AC° as already pointed out in the proof of Corollary |5] The required 
AC° computation in input (a;, y) now checks that its second input argument is a 
correct path of the corresponding machine on input x] if so it outputs 1 iff this 
path is accepting and 0 otherwise. If y does not encode a correct path then the 
neutral letter is output. □ 

A careful inspection of the just given proof reveals that the result not only 
holds for language B G Af, B C {0, 1}*, but also for languages B that are 
obtained from some B' G Af, BCE* (possibly |A| > 2) by block encoding. The 
same generalization holds for all results that we state below for “B G Af" (i.e., 
Theorem ETH and all results in Sect. ED- 

In the context of NC^ and subclasses, some interesting results can be obtained 
for classes of the form (-('“^AC^. 

First, Barrington’s theorem |Bar89j yields: 

Theorem 15. (55)^°«AC° = NC^ 

Theorem 16. 1. = NC^ for every regular language B whose syntac- 

tic monoid is non-solvable. 

2. (SOLVABLE)i°sAC° = ACC°. 

Generally the class (i?)^°sAC° roughly corresponds to AC° circuits with a B 
gate on top, e.g. (maj)'°®AC° is the class of all languages accepted by percept- 
rons. 

AC° circuits with arbitrary B gates are examined in |BIS90IBI94j (see also 
Sect. 0. 
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4.2 Logspace and Logtime Leaf Languages 

In the same spirit as above for nondeterministic polynomial time machines, Jen- 
ner, McKenzie, and Therien examined in [JMT94j leaf languages for nondeter- 
ministic logarithmic time and logarithmic space machines. 

First turning to the logspace case, we observe that the trivial way to formulate 
Leaf^(S), the class defined by logspace machines with leaf language B, as a, 
class does not work (L denotes the class of logspace decidable sets). This 
is because (for B G Af) already (i?)PP = (i?)PAC° (see Sect. 14.111 . and therefore 
also (B)PP = {B)PL. 

However, if we turn to logarithmic space-bounded one-way protocol machi- 
nes or 2-1-machines |Lan86j . we can come up with a connection. A 2-1-Turing 
machine is a Turing machine with two input tapes: first a (regular) input tape 
that can be read as often as necessary, and second, an additional (protocol) tape 
that can be read only once (from left to right). Define 2-1-L to be the class of all 
two argument languages L that can be computed by logspace-bounded 2-1-TMs 
such that in the initial configuration, the first argument of the input is on the 
regular input tape, and the second argument is on the one-way input tape. Then 
the following can be shown using ideas from [Lan86j : 

Theorem 17. Let B C {0, 1}*. Then {B)p 2-1-L = Leaf^(H). 

Jenner, McKenzie, and Therien showed that in a lot of cases, the balanced 
and unbalanced model coincide for logarithmic space machines, and moreover it 
sometimes coincides with the polynomial time case, e.g. Theorem |5] above also 
holds with leaf languages for logspace machines. Interesting to mention is that 
in the logarithmic space model, regular leaf languages define the class P, while 
NC^ defines the class P SPACE. 

In the logarithmic time case, coincidence with the logarithmic time reduci- 
bility closure could be shown for all well-behaved leaf languages. Formulated in 
terms of our quantifier, some of their results read as follows: 

Theorem 18 ( [JMTQ^ L 

1. (REG)'°sdLOGTIME = NC^ 

2. (CFL)'°sDLOGTIME = LOGCFL. 

3. (CSL)^°8DLOGTIME = PSPACE. 

Theorem 19 ( |JMT9^ ~). 

1. (H)*°SDLOGTIME = NC^ for any regular language B whose syntactic mo- 
noid is non-solvable. 

2. (SOLVABLE)^°sDLOGTIME = ACC°. 

3. (APERIODIC)i°sDLOGTIME = AC°. 
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4.3 Other Models 

Type 2 Operators Operators ranging not over words but over oracles, so called 
type 2 operators, have been examined in | BW96IBVW96IVW97| and elsewhere. 
Most of the considered classes coincide with classes of the form {B)^IC where 
/C = coNP or K. — PSPACE and T is the class of all exponential time computable 
functions (let us write as a shorthand for this choice oi T). A word of 

care about the computational model however is in order now. We say that a 
language L belongs to the class (B)®’'PcoNP if there is a function / computable 
in exponential time, and a set A such that x € L <;=^ xa„ [0 . . . f{x)] G B, 
where A is accepted by some co-nondeterministic Turing machine M that on 
input {x, y) runs in time polynomial in the length of x. The length of y is possibly 
exponential in the length of X] thus to enable M to access all positions of y within 
its time bound we supply M with a regular input tape on which x is found, and 
a second input tape for y, which is accessed by an index tape . This special 
input tape is similar to an oracle tape, and therefore quantifiers over strings on 
this tape translate to quantifiers over oracles. (In the case of (B)®’'PPSPACE we 
require our machines to use space no more than polynomial in the length of their 
regular input x.) 

Theorem 20. Let B G M . Then we have: 

(B)®’^PEXPTIME = (B)®='PPSPACE = {BY^'^coNP. 

Proof sketch. If we look at the proof of Theorem [H]we see that to check cor- 
rect computation paths we actually don’t need the full power of AC°. is 
sufficient, but we have to modify the computation model slightly as follows: The 
log-time machine has a regular input tape (which is accessed as usual by using 
an index tape) and a second input tape on which the path to be checked is given 
(again access is by an index tape). We thus get: 

(B)PP = (B)PAC° = (B)Pil}°s. 

Using standard translation arguments we now get the claim of the theorem by 
lifting up this equation one exponential level. □ 



NC^ Leaf Languages In jCMTV9^ leaf languages for nondeterministic fi- 
nite automata were considered. The original input is however first given into a 
uniform projection, and the result of this projection is then fed into the NFA. 
Barrington’s Theorem [T^ implies that with regular leaf languages we thus get 
exactly the class NC^. Some other characterizations were given in [CMTV9^ . 
and the model was also used to examine counting classes within NC^. 

Function Classes In |KSV97j the definability of function classes has been 
examined. An oracle separation criterion generalizing Theorem 0 was given and 
applied successfully in some open cases. 
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5 Leaf Languages vs. Lindstrom Quantifiers 

Lindstrom quantifiers [ILin66j are a well established generalized quantifier notion 
in finite model theory. The reader probably has noticed some resemblance of our 
definition in Sect. [2] with that of Lindstrom quantifiers. It will be our aim in the 
upcoming sections to make this precise. 

As we will see there is a strong connection between leaf languages for poly- 
nomial time machines and second-order Lindstrom quantifiers. Since this notion 
might not be so well-known, we give — after very briefly recalling some termino- 
logy from finite model theory — a precise definition in Sect. 15.11 

In later subsections we will have the need to talk about the second-order 
version of a given first-order Lindstrom quantifier. We chose to make this precise 
by talking about the semantics of quantifiers given by languages instead of the 
usual way of defining semantics by classes of structures. In the next subsection, 
we will define how a language B gives rise to a first-order quantifier and a 
second order quantifier Qg. 

5.1 Second-Order Lindstrom Quantifiers 

A signature is a finite sequence r = (i?i, . . . , iZfe, Ci, . . . , Cf) of relation sym- 
bols and constant symbols. A finite structure of signature r is a tuple A = 
(A, i?^, . . . , R-^, c ^, . . . , c;^) consisting of a finite set A (the universe of A) and 
interpretations of the symbols in r by relations over A (of appropriate arity) 
and elements of A. Struct ( t) is the set of all finite ordered structures over r. 
The characteristic string XR of a relation R G {0,...,n — 1}“ is the string 
XR =def bi ■ ■ ■ bn<^ where 6^ = 1 iff the Ath vector in {0, . . . , n — 1}“ (in the order 
(0, . . . , 0, 0) < (0, . . . , 0, 1) < (n — 1, . . . , n — 1, n — 1)) is in i?. For 1 < f < n“, 
let denote the i-th. bit in xr- 

If £ is a logic (as e.g. FO or SO) and /C is a complexity class, then we say that 
£ captures K if every property over (standard encodings of) structures decidable 
within K. is expressible by £ sentences, and on the other hand for every fixed £ 
sentence (j), determining whether A \= 4> can be done in /C. As an abbreviation 
we will most of the time simply write K, = C. 

A first-order formula with k free variables defines for every structure A the 
relation =def {a G | ^ h '/'(“) }; see |EF95| . 

Every class of structures K C Struct(cr) over a signature tr = (Pi, . . . ,Ps) 
defines the first-order Lindstrom quantifier Qk as follows: Let fii, . . . , fis be 
first-order formulae over signature r such that for 1 < t < s the number of free 
variables in is equal to the arity of P^. Then 

QkXi, ...,Xs . . . , fisiXs)] 

is a QkFO formula. If A G Struct(r), then 

A [= Q KXi ^ . . . T Xg [0i(xi), . . . , (fs (^s )] 
iff {A,fif,...,fif)GK. 
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The just given definition is the original definition given by Lindstrom [ILin66j . 
which the reader will also find in textbooks, see e.g. |Ebb85IEF95| . For our 
examinations, the following equivalent formulation will be useful (observe that 
this only makes sense for ordered structures): 

Given a first-order formula (j) with k free variables and a corresponding finite 
ordered structure A, this defines the binary string of length (n = |A|). 
Now given a sequence 4 >i, . . . , (j)s of formulae with k free variables each and a 
structure A, we similarly get the tuple , X^^), where Ix^^^l = ••• = 

|x<)1-a| = n^. Certainly, there is a one-one correspondence between such tuples 
and strings of length over a larger alphabet (in our case with 2® elements) 
as follows. Let Ag be such an alphabet. Fix an arbitrary enumeration of Ag, 
i.e. Ag = {oo, oi, ..., 02^-1}. Then corresponds to the string 

61&2 ■ ■ ■ bnk, where for 1 < i < n^, G Ag, = Ofc for that k whose length s 
binary representation (possibly with leading zeroes) is given by X^f[i] ' ‘ ‘ X<t>-^[i]- 
In symbols: Wg{x^A , . . . , x^^) = &1&2 

This leads us to the following definition: A sequence [^1, . . . , 4 >g] is in first- 
order word normal form, iff the (pi have the same number k of free varia- 
bles. Let r be an alphabet such that |T| > 2®, and let B C F*. Then A ^ 
QBx[(pi{x),...,pg{x)] iff Wg{x^A,...,Xci>^) e B. 

It can be shown | BV96IBur96| that every Lindstrom quantifier Qj^ can with- 
out loss of generality be assumed to be of the form Qb &s just defined. This is 
the case since for every sequence [pi, , pg\ of first-order formulae we find an 
equivalent sequence in word normal form such that the corresponding formulae 
with Lindstrom quantifier express the same property. 

Second-order Lindstrom quantifiers are defined as follows [BV9fi|Riir9fij : Gi- 
ven a formula p with free second-order variables Pi, ... , Pm and a structure A, 
define p'^'^ =def { > Rm) \ -4 h P{Rt, ■■■, Rm) }. and let x^-^^ be the 

corresponding characteristic string, the order of vectors of relations being the 
natural one induced by the underlying order of the universe. If the arities of 
Pi, ... , Pm are ri, . . . , rm, resp., then the length of X,^2-4 is 2” ™ 

Let (T = {oi, . . . ,(jg) be a signature, where (Ti = {Pi^i, . . . , Pi^mt) for 1 < 
i < s. Thus cr is a signature consisting of a sequence of s signatures with only 
predicate symbols each. Let £ij be the arity of Pij. A second-order structure 
of signature cr is a tuple A = {A,TZi, . . . ,TZg), where for every 1 < i < s, 
R-i F { {Ri,i^ • ■ ■ j Ri,mi) I Ri,j F Given now a signature r and second- 

order formulae pi{Xi), . . . , pg(Xg) over r where for every 1 < i < s the number 
and arity of free predicates in pi corresponds to ai. Let /C be a class of second- 
order structures over a. Then QicXi, . . . , Xg [pi{Xi), . . . , </)s(Xs)] is a QicSO 
formula. If A S Struct(r), then A h QkXi, . . . , X g[Pi{Xi), . . . ,Pg{X g)] iff 
{A,pf,...,pf)€lC. 

Again, we want to talk about second-order Lindstrom quantifiers defined by 
languages. Thus we define analogously to the above: A sequence [pi{Xi), . . . , 
<()s(ACs)] of second-order formulae is in second-order word normal form, if the 
pi, ... ,ps have the same predicate symbols, i.e. in the above terminology cti = 
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••• = as = (Pi,..., Pm)- Let for 1 < i < m the arity of Pi be r^. Observe 
that in this case, |Xj 2 .a| = ••• = |xm.A| = (for n = Ml), thus 

{x^ 2 -a , - - - , ) corresponds to a word of the same length over an alphabet of 

cardinality 2®. Given now a language B C P* with |P| > 2®, the second-order 
Lindstrom quantifier given by B is defined by M |= [c^i(X), . . . , <();,(X)] iff 

Ws{X^2A,...,X^2a) G B. 

Again it was shown in [BV96IBur96| that for every second-order Lindstrom 
quantifier Q/c there is an equivalent Q^. 

When talking about the first-order Lindstrom quantifier given by B, we so- 
metimes explicitly write instead of Qb- In addition to the above logics 
Q^FO and Q)jSO where we allow Lindstrom quantifiers followed by an ar- 
bitrary first-order (second-order, resp.) formula, we also need Q^FO (where we 
have a second-order Lindstrom quantifier followed by a formula with no other 
second-order quantifiers), and FO(Q^) and SO((5)j) (where we have first-order 
(second-order, resp.) formulae with arbitrary nesting of universal, existential, 
and Lindstrom quantifiers). For a class of languages C we use the notation Qc 
with the obvious meaning, e.g. FO(Qc) denotes all first-order sentences with 
arbitrary quantifiers for B £ C. 

5.2 A Logical Characterization of the Leaf Concept 

The main technical connection between polynomial time leaf languages and 
second-order Lindstrom quantifiers is given in the following theorem: 

Theorem 21. Let M be a polynomial time nondeterministic machine whose 
computation tree is always a full binary tree, and let B C {0, 1}*. Then there is 
a Si formula (f such that 

Leaf^^(P) = Q],X [</)(X)] . 

Proof sketch. We use a modification of Fagin’s proof |Fag74| . The quan- 
tifier will bind the nondeterministic guesses of the machine. The second-order 
quantifiers in (j) will bind variables Y that encode computation paths of M. The 
formula 4>{X) says “there is a T encoding a correct computation path of M 
corresponding to nondeterministic guesses X, which is accepting.” □ 

If we deal with B C P* not necessarily over the binary alphabet, then instead 
of 4> above, we get formulae 4>s, ■ ■ ■ ,<l>s such that 

Le^i^{B) = Q],X[MX),...,MX)]- 

4>i{X) says “there is a T encoding a correct computation path of M correspon- 
ding to nondeterministic guesses X, and the leaf symbol produced on this path 
has a 1 in bit position i (in binary). Thus what we have here is some block- 
encoding of P in binary strings of length s. 

The just given theorem shows that FBTLeaf^(P) C Q^Sl. The question 
now of course is if there is a logic capturing FBTLeaf^(P). For the special case 
B £ Af, the answer is yes. 
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Theorem 22 ( [BV96] h Let B €j\f. Then Q^FO = BLeaf^(S). 

Proof sketeh. This time Qg binds the nondeterministic guesses X as well as the 
encoding T of a possible computation path. The first order formulae “output” 
the neutral letter, if Y does not encode a correct path. This proves the direction 
from right to left. For the other inclusion, we observe that we can design a Turing 
machine which branches on all possible assignments for the relational variables 
and then simply evaluates the first-order part. □ 

In the preceding theorem BLeaf^(i3) is captured by the logic Q^jFO uni- 
formly in the sense of |MP93IMP94j : this means that the particular formula 
describing the Turing machine is independent of the leaf language. 

Let us next address the question if the quantifier in the preceding theorem is 
genuinely second-order. First, we have to give some definitions. A succinct repre- 
sentation |Wag86a|BLT92IVei96| of a binary word a; is a boolean circuit giving 
on input i the ith bit of x. The succinct version sA of a language A is the follo- 
wing: Given a boolean circuit describing a word x, is a: = X 1 OX 2 O • • • Xn-iOxnlw 
for arbitrary w € {0, 1}*, such that X\X 2 • • • x„ S A? The boolean circuits we al- 
low are standard unbounded fan-in circuits over AND, OR, NOT. The encoding 
consists of a sequence of tuples (g,t, h), where g and h are gates, t is the type 
of g, and h is an input gate to g (if g is not already an input variable). 

Now we see that there is an equivalent first-order logic for Q)jFO. 

Theorem 23. Let B gM. Then BLeaf^(R) = Q^jFO = Q°bFO. 

Proof sketeh. Veith [Vei96j showed that sB is complete for BLeaf^(i?) under 
projection reductions. (A somewhat weaker result appeared in |BL96j l. This 
together with Theorem 1221 implies the theorem. □ 

5.3 Applications 

Burtschick and Vollmer in [IB V9fi] also examined logically defined leaf languages. 
It turned out that if the leaf language is given by a first-order formula, then the 
obtained complexity class is captured by the corresponding second-order logic. 
More specifically, they proved for instance: 

Theorem 24 ( [BV96J L Let B G Af . Then {Q%E°)PP = 

As a special case of Theorem we get a characterization of the classes of 
the polynomial hierarchy which is tighter than the one in Theorem 

Corollary 7. (r°)PP = 

^From the PSPACE characterization Theorem |2] and the above results, we 
get the following model-theoretic characterization of PSPACE: 

Corollary 8. Q^,,FO = Q°s,FO = PSPACE. 
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5.4 First-Order Quantifiers 

It is known from the work of Immerman et al. [Imm89lBIS90] that (uniform) 
AC° is captured by FO. However, for this result, we have to include the bit 
predicate in our logic. We make this assumption throughout this subsection (all 
the previously given results are valid without the bit predicate). 

Theorem 25. Let B C {0, 1}*. Then (H)'°sAC° = Q^FO. 

Theorem |25] together with results from Sect. 14.21 on logtime leaf languages, 
gives some more model-theoretic characterizations. 

Corollary 9. 1. PSPACE = Q^slFO = FO(Qcsl)- 
g. LOGCFL = QVFO = FO(gcFL). 

Proof sketch. One can show that generally Leaf^"^(H) C Q^FO. The corollary 
then follows from Theorem [18] □ 

6 Conclusion 

We examined a generalized quantifier notion in computational complexity. We 
proved that not only all quantifiers examined so far (whether in the logarithmic, 
polynomial, or exponential time context) can be seen as special cases of this 
quantifier, but also circuits with generalized gates and Turing machines with 
leaf language acceptance. 

Most of the emerging complexity classes can be characterized by means from 
finite model theory. We gave a precise connection to finite model theory by 
showing how complexity classes defined by the generalized quantifier relate to 
classes of finite models defined by logics enhanced with Lindstrom quantifiers. 

A number of questions remain open. The results we gave in Sect. O related 
complexity classes to logics of the form “Lindstrom quantifier followed by a 
usual first- or second-order formula.” It is not clear if logics defined by arbitrary 
nesting of Lindstrom quantifiers have a nice equivalent in terms of the generalized 
complexity theoretic quantifier. Barrington, Immerman, and Straubing proved: 

Theorem 26 ( [BIS90] 1. Let B G M. Then FO{Q^^) = AC*^[H] (AC° circuits 
with B gates). 

Moreover one can show: 

Theorem 27 ( [Vol96b] ) . Let B € M. Then FO(Q)j) is the oracle hierarchy 
given by (H)pAC'' as building block. 

But the general relationship remains unclear. The work of Makowsky and 
Pnueli (see |MP93IMP94| 1. Stewart (see e.g. jSte91ISte92j l. and Gottlob (see 
[IGot95p shows that there is a strong relation between Lindstrom logics and re- 
lativized computation. The just mentioned results also hint in that direction. 
Gottlob [Got95| related the expressive power of logics of the form “Lindstrom 
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quantifier Q followed by first-order formula” to the expressive power of FO((5). 
However his results only apply for superclasses of L (logarithmic space) . Inte- 
resting cases within NC^ remain open. Generally the connection between prenex 
Lindstrom logics vs. logics allowing arbitrary quantifier nestings on the model 
theoretic side, and leaf languages vs. oracle computations on the complexity 
theoretic side should be made clearer. 

It is open for which of the results in Sect. 15. 41 the bit predicate is really needed. 
One can show that without bit, QcflFO = CFL contrasting the corresponding 
result with bit given in CorollaryEl The power of the bit predicate in this context 
deserves further attention. 

From a complexity theoretic point of view, we think the main open question 
is the following. A lot of classes defined by leaf languages have been identified. 
However, most of the results are not about singular leaf languages but about 
classes of leaf languages. For example (see Theorem |3), if we take an arbitrary 
aperiodic leaf languages, then the complexity class we obtain is included in 
PH, and conversely we get all of PH when we allow aperiodic leaf languages: 
BLeaf^ (APERIODIC) = PH. The question now is the following: What exactly 
are the classes of the form BLeaf^(B) for aperiodic B1 Is it possible to come up 
with a complete list of classes that can be defined in this way? Some of the results 
in Sect . IXD point in this direction. For example we know that there is no class 
between P and NP that can be defined by a regular leaf language (unfortunately 
the result given in Sect. 13.41 holds only for the unbalanced case). Can we come up 
with similar result for the balanced case? Generally, very little is known about 
the power of single leaf languages as opposed to classes of leaf languages. 

Acknowledgment. For helpful discussions I am grateful to J. Makowsky (Haifa) 
and H. Schmitz (Wurzburg). 
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1 Introduction 

In recent years, generalized quantifiers (see [H3j ) have received quite a lot of novel 
interest because of their applications to computer science and linguistics. Their 
definability theory has made considerable progress during the last decade, which 
will be the subject of the next section. The proofs of many of these results often 
use results of Ramsey theory, such as theorems of van der Waerden and Folkman, 
and yet, the answers to some of the definability problems seem obvious from the 
outset. This raises the natural question whether Ramsey theory is really needed 
in the proofs (cf. [vBWj i or whether easier ways of proof might be discovered. 
The purpose of this paper is to argue in favour of the former and to convince 
the reader of the cruciality of Ramsey theory for quantifier definability theory. 

The task of showing the necessity of Ramsey theory for quantifier theory does 
not translate easily to a rigorous mathematical problem, so the formulation of the 
framework is itself a problem. Roughly, we want to find an undefinability result 

V concerning quantifiers and a combinatorial theorem K, and then show that 

V ^ 1C. We may as well fix that /C is van der Waerden’s theorem, whose variants 
have occured frequently in the recent quantifier literature. However, since V 
and /C are both theorems of ZFC, the statement V ^ 1C holds trivially, which 
is not really what we meant. This kind of difficulties are usually overcome in 
reverse mathematics by the metamathematical change to weaker axiom systems. 
However, I have abandoned this approach in order to preserve the clarity of the 
exposition. The reason is that generalized quantifiers are complex objects and 
the formulation of the basics of their theory in a weak axiom system would 
require a lot of tedious technical work. 

Instead, a more combinatorial way of arguing is used. At first, a simple proof 
of P /C is presented (in Section 4), which should already convince a full- 
fledged pragmatic. In essence, this proof shows that certain function related 
to quantifiers grow faster than van der Waerden’s function W, which will be 
elaborated in Section 5. What exactly is meant, becomes more apparent as we 
proceed. 
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We have not yet paid any attention to the choice of the undefinability result 
V. Clearly it is not reasonable to allow V to be any quantifier result whatsoever, 
since it may be possible to code extremely difficult mathematical problems as 
quantifier problems; we should rather try to find a result V relevant for the 
field. As a result, we are not ready for the choice of V until in Section 4. Before 
that, we shall have a look at the problematics of quantifier definability theory 
in Section 2. We shall find out that we may as well restrict our considerations 
to monadic quantifiers, whose theory is sketched in the Section 3. The point is 
that, in the case of monadic quantifiers, the definability questions can be reduced 
to colouring problems about relations. These, in turn, give rise to certain fast- 
growing functions which give an upper bound for the van der Waerden’s function. 

2 Quantifier Definability Theory 

Although 40 years has passed since Andrzej Mostowski presented his notion 
of a generalized quantifier, systematic treatment of definability problems of 
quantifiers is a fresh research subject. Up to the mid-1980’s, the emphasis of 
the research in generalized quantifier theory was on finding logics with good 
model-theoretic properties. Definability and undefinability results were merely 
by-products of this study, e.g., Keisler’s proof m that does not have 

the ^-interpolation property was based on the following hidden result: 

Theorem 1. (Keisler) Qf is not definable in C^uj{Qi). □ 

To explain the quantifiers involved, let us introduce some notation. For a 
class of cardinals S, let Cs be the quantifier with defining class 

Kcs = { M e Str({[/}) I \U^\ e S} 

where C/ is a unary relation symbol. In other words, for every { C/}-structure fOl 
we have 

m h Csx U{x) ^ e S. 

Quantifiers of form Cs are called eardinality quantifiers. Similarly, Es is the 
quantifier whose vocabulary is {E} with E binary, and defining class Kes is the 
class of all structures DJI G Str({A}) such that E^ is an equivalence relation 
with the number of equivalence classes k belonging to S. Clearly Cs is definable 
in Cuiuj{Es) by the sentence 



Esxy {U{x) Ax = y). 

Now Qi = Cs and Qf = Es where S is the class of all uncountable cardinals. 
Hence, Keisler’s result shows that Es is not necessarily definable in Cujuj{Cs). 
His counterexample to ^-interpolation of Cuuj{Qi) was later generalized by Xa- 
vier Caicedo jCa 2]. This, in turn, inspired some quantifier definability theory in 
finite model theory related to quantifiers similar to Es by Flum, Schielen and 
Vaananen [FSV| . 
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As it has become clear, the simplest form of a definability problem is the 
following: 

(*) Is (5 definable in the logic 

where Q is a generalized quantifier and Q is a set of such. Most of the papers 
on generalized quantifiers where this is the main motivation are less than fifteen 
years old. One of the earliest is the paper by Luis Jaime Corredor US] solving 
completely the definability problems amongst cardinality quantifiers. Denote by 
0 the common extension of integer and cardinal addition such that k 0 n = 
n 0 K = K for every infinite cardinal k and n G 1 a. For S a class of cardinals and 
n€l, let S(Bn = {K(Bn\KGS}. 

Theorem 2. (Corredor) Let S and Ti, i G I, be classes of cardinals. Then the 
following are equivalent: 

1) Cs is definable in | * S / }). 

2) There is a (finite) Boolean combination T of classes Ti 0 n, i G I and n G 1, 
such that |S'Z\T| < u>, i.e., the symmetric difference of S and T is finite. □ 

Observing that 3 = Ce where E is the class of non-zero cardinals, case 2 can 
actually be replaced by 

2’) S' is a Boolean combination of classes T^ 0 n and T 0 n where i G I and 
n G Z. 

In the hindsight we may say that the neat formulation of the preceding theorem is 
possible because cardinality quantifiers enhance the expressive power of the first 
order logic by expressing things about extremely simple structures: the defining 
class of a cardinality quantifier consists of structures whose vocabulary contains 
only one predicate that is unary. In the same vein, it is reasonable to ask if 
theoremCjis only an instance of a more general result, i.e., since the vocabulary 
of Qf is {E} with E binary and that of Qi is U with U unary, could it be 
possible that Qf is not definable by any set Q of cardinality quantifiers. As it 
happens, this is exactly the case, and this is not accidental. For Q a quantifier 
with vocabulary r, let the arity of Q be 

ar(Q) = sup { nii \ R G t} 

where ur is the arity of the relation symbol R G t. A quantifier Q is monadic, if 
ar(Q) = 1, binary, if ar(Q) = 2, and ternary, if ar(Q) = 3. The collection of all 
quantifiers Q with arity at most n is denoted by Q„. Then we have the following 
result due to Bella |Hlj and implicitly by Caicedo |Calj . 

Theorem 3. (Caicedo, Hella) Qf is not definable by monadic quantifiers, i.e., 
in the logic T(.ja;(Qi)) nor even in Toow(Qi)- LI 

Even better, after the preliminary results of the other people (e.g., [VT]) 

Hella was able to establish the following hierarchy result, among other results. 

Theorem 4. (Hella) For every n G oj and non-zero ordinal a, the Magidor- 
Malitz quantifier is not definable in the logic Coou{Q,n)- L 
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The definition of is omitted; an interested reader may consult the original 
paper m or the survey |HL|. 

At that time in the late 1980’s, the focus in the research of generalized quan- 
tifiers was rapidly shifting towards finite structures. Hella’s methods, certain 
model-theoretic games, were easily adaptable in the new context, as shown in 
the paper |H2| . Kolaitis and Vaananen | |KV| did some systematic study in the 
realm of monadic simple quantifiers. A quantifier Q is simple, if its vocabulary 
consists of a single relation symbol. The Hdrtig quantifier, or the equicardinality 
quantifier, I, is an example of a monadic quantifier that is not simple; it is the 
monadic quantifier binding two formulas with the defining class 

Ki = {OTg Str({C/,F}) I \U^\ = \V^\ }. 

The Rescher quantifier R has the same vocabulary as I, but its defining class is 

Kr = {OTg Str({17,F}) I \U^\ < \V ^\ }. 

I is easily definable in C^^{R) by the sentence 

Rxy{U{x),V{y)) A Rxy{V{x),U{y)). 

Kolaitis and Vaananen proved the following: 

Theorem 5. (Kolaitis and Vaananen) Let Q he a finite set of simple monadic 
quantifiers. Then: 

a) I is not definable in 

b) R is not definable in U {/}). 

c) A 2 N is not definable in Cuiuj{Q U {/}) where 2N is the set of euen natural 
numbers. 

These statements hold euen if restricted to finite structures. □ 

A notable feature in the proof is that it rests on the following results of 
Ramsey theory: 

van der Waerden’s Theorem. |vW] For every k,t G N, there is w G N such 
that if the set {0, . . . ,w — 1} is coloured with at most t colours, say, by the 
colouring x- {0, ... ,w — 1} ^ F with |F| < t, then there is a monochromatic 
arithmetic progression of length k, i.e., there are a,d G N, d 0, such that 
a+ (k — l)d < w and %(a) = x(a -I- id), for every i = 0, . . . ,k — 1. □ 

Folkman’s Theorem. For every k,t G N, there is n G N such that for every 
colouring y: {0,... ,rc— 1}— with at most t colours (i.e., |F| <t), there exist 
K C N of size \K\ = k such that ^or all non-empty I, J <Z K, 

we have x(X*e/*) = x(Ejejj)- ° 

These theorems give rise to combinatorial functions, e.g., van der Waerden’s 
function W : N x N — >■ N where for every G N, W{k,t) is the least w G N 
satisfying the condition in the theorem. We also denote W 2 : N — >■ N, W 2 {n) = 
W{n,2). 
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The reason why finite model theory pushed combinatorics in the front-line lies 
in the nature of the model constructions. While infinite structures can be made 
closed under different kinds of conditions and combinatorics remains hidden in 
the structures, this is usually not possible when constructing finite structures. 
This means that explicit use of combinatorial principles is required. In the se- 
quel, Ramsey theory became a common tool in quantifier definability theory. 
The hierarchy result |4] inspired search for refinements of the hierarchy. In par- 
ticular, linguistic interest (see m) seems to have spurred research on monadic 
quantifiers. The width of the quantifier Q is just the cardinality of its vocabu- 
lary T, in symbols, wd(Q) = |r|. Various researchers ([CI]. [NV| .[Luj) obtained 
independently the following monadic hierarchy result: 

Theorem 6. (Lindstrom, Nesetfil and Vaananen, Luosto) For every n G N, 
there is a monadic quantifier of width n + 1 which is not definable by means of 
monadic quantifiers of width at most n, i.e., in any Tc^c^(Q) where Q is a set of 
monadic quantifiers of width at most n. □ 

The explained involvement in combinatorics caught the eye of Johan van 
Benthem and Dag Westerstahl. They end the section 3 of their paper jvHWj by 
a short speculation of the need of the Ramsey theory. After a short description 
of the results and the methods in m, they write: 

’’This leads to the question whether every proof of the definability result 
requires Ramsey theory?” 

And later: 

” Given the very general nature of generalized quantifiers it may be worth- 
while to do some ’reverse mathematics’ in the field of finite combinatorics 
and definability questions, and thus to assess the combinatorial content 
of certain results about generalized quantifiers.” 

In order to be able to answer to these demands, we need to sketch the basics of 
monadic quantifier definability theory from [E^ in the next section. 



3 Reduction to Combinatorics 

In this section some basics of the monadic quantifier definability theory are 
described, which is a prerequisite for the analysis of the combinatorial contents of 
undefinability results. The technicalities are kept aside, so that the presentation 
is necessarily sketchy and appeals a lot on the intuition of the reader. (The results 
and exact definitions can be found in Eu].) Something concrete is needed for 
the further treatment, though, and this core of the theory is condensed in the 
notion of the irreducibility of a relation in the end of the section. 

To start with, we note that monadic structures are easily describable. A 
structure with three unary predicates, say 9JI G Str({F, Q, F}), is drawn in the 
Figure 1 below. 
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Q 



Figure 1 



The structure 911 is uniquely determined if we count the number of elements 
in each part, i.e., by the tuple (kq, . . . ,Kr). In general, a monadic structure 9H 
for a finite vocabulary r, say |r| = I, can be identified by a tuple Kgjt of length 
2* of cardinal invariants. In other words, if 911,91 € Str(r), then 9H and 91 are 
isomorphic iff Rm = K<n- 

Recall that for each cardinality quantifier Q there is a class S of cardinals such 
that Q = Cs- Analogously, each monadic quantifier Q with finite vocabulary can 
be reduced to a relation 



nQ) = {rw\^&kq}. 



Example 1. The vocabulary of the Hartig quantifier is {{7, R} and for 9U G 
Str({{7, V}), we have 

MeKi ^ \U^\ = \V^\. 

To describe 9H G Str({[7, R}) up to isomorphism, we count that there are kq.ot 
elements in the intersection of the predicates, U^DV^, Ki^m elements in \ 
V^, H 2 ,m elements in \ and elements outside the predicates 

and (note that we have to fix some order in which to enumerate the 
invariants, but it is immaterial which particular order, as soon as it is the same 
for all structures). Therefore, 

KOT = (Ic/®' n |C/®^ \ \ C/®^|, |Dom(91t) \ U 



and 

Hence, 



TIgKi 



kq, m ® = kq, OT © K2,m- 



TZ{I) = { ( ko , Ki,K2, Ks) I Ko (B Ki = Ko (B K2 }■ 
Note that for finite 9U G Str({[/, V}) we have the simple relation 



m&Ki 



Him = H 2 M- 
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Similarly, for the Rescher quantifier we get 



TZ{R) = { (ko, Ki, K2, K3) I Ko © Kl < Ko © K2 } 



and 

n = { (mo, mi, m2, m-3) G | mi < m2 }. 

There is a result analogous to Theorem ^ for cardinality quantifiers that 
characterizes the definability among monadic quantifiers. In order to explain the 
notation, let us consider the structure 2t from Figure 1 and the sentence 

ip = Ixy{P{x) V Q{x),R{y)). 

Then 21 |= (/j iff | U Q® | = | i?® | iff the interpreted structure is in Kj for 
which = P'^U Q® and = R^. 




Figure 2 

It is easily seen that itgjt = {k 4 (B kq(B K 7 , kq (B K 2 ® K 3 , K 5 , ki). Schematically 
we get Kgjt from the components of Sa when we sum over the set of indices 
{4,6,7}, {0,2,3}, {5} and {!}, each in turn. In symbols, we write 



Km = s{k<2i,U) 



where U = ({4, 6, 7}, {0, 2, 3}, {5}, {!}). Then we have 21 ^ iff s{R, U) G 72.(7). 

Similarly to Theorem [2 we have to take translates into account. Below Jn^i 
stands roughly for the set of all meaningful pairs {U,n) when we reduce the 
size of the tuple from n to I . Reformulating a result of Vaananen [V2] in our 
notation, we get: 

Theorem 7. (Vaananen) Let Q he a monadic quantifier and Q a set of monadic 
quantifiers, all with finite vocabularies. Let n = and Iq = for each 

q G QU {3}. Then the following are equivalent: 

1) Q is definable in CuiuiiQ)- 
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2) 'R-{Q) is a Boolean combination of relations of form 

{ R I s{R, J7) © n G 'R-(q) } 

where g G Q U {3} and {U , n) G Jn,i, ■ ^ 

It is noteworthy that if we replace TZ{Q) by R and TZ{Q)’s by some Sfs in 
case 2, what is left is a totally combinatorial condition on relations. This kind 
of consideration led to the notions of the rank r{R) of a relation R and the 
relative rank r_|_(i?) of a relation relative to a monoid {M, +) in | Lu| . The former 
is simpler, but the latter corresponds better to the intended application. 

A quantifier Q with vocabulary r is called universe-independent, if we have 
21 G Kq iff 03 G Kq whenever 21, 05 G Str(r) are such that for every i? G r, it 
holds that i?® = i?®. Then the following hold: 

Theorem 8. Let I G N and k be an infinite cardinal and C be the set of cardinals 
below K. Let Q be a monadic quantifier such that n = G N. Then the 

following are equivalent: 

1) Q is definable by monadic universe-independent quantifiers of width at most 
I on structures of cardinality less than k. 

^;re(7^(Q)nC”) <2'-l. □ 



Proposition 1. Let R C C" be a relation where C is a set of cardinals. Then 
If C is infinite, ClTN= {0} and r{R) > 3, then r,^{R) = r{R). 

□ 



Let me sketch one application of these combinatorial results. Every quantifier 
has liftings which are called resumptions. For example, the second resumption 
of the Hartig quantifier is the quantifier L^^'> with the vocabulary {R, S'} where 
R and S are binary and such that for every DJI G Str({i?, S|), 

971 ^ I^'^'^xy, tu{R{x, y), S{t, u)) 
iff 

i.e., if there are as many pairs in Rp^ as there are pairs in S®^ Note that even if 
the definitions of / and have the same appearance, is a binary quantifier 
rather than monadic, since it binds two variables in both formulas. 

Westerstahl posed, among other problems, the question if is definable in 
terms of monadic quantifiers of finite width. The question was answered to the 
negative in IS]- Indeed, suppose to the contrary that there is a finite set Q of 
monadic quantifiers of finite width such that L^‘^'> is definable in Without 

loss of generality, these quantifiers are universe-independent. Pick to G N such 
that TO > wd(Q), for every Q G Q. Let n = 2™. For q G Z", consider the relation 

= { a; G N” I a; 0, q and x orthogonal }. 
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(Here, orthogonality is just the usual notion of linear algebra, i.e., q and x are 
orthogonal iff 5 • a: = 0.) Then it can be shown that if Qq is the quantifier with 
TZ{Qq) — Rq, then Q is definable in and therefore also in To 

demonstrate this by an example, consider q = (0,3, —2,0). Supposing that the 
vocabulary of the quantifier is {U,V} and the indices as in Example [1] put 

3 yo, yi-,V2 3 to , h = yi A ~^yo = y2 A ~^yi = y2 A -ifo = ti A 



Then for every DJI S Str({17, E}), 971 ^ iff OH S Kq. provided that there are 
at least three elements in 971. 

Now by the previous theorem, we have r<^{Rq) < O'" — 1 < n. On the other 
hand, it may be shown that there are vectors q such that r(^{Rq) = n, which is 
a contradiction. 

The notion of irreducibility is derived from the concept of the rank, not of 
that of relative rank. The preceding results show that the rank works well if we 
consider infinite structures. Since the emphasis of today’s research is on finite 
structures, though, we might have a problem. The explanation why irreducibility 
is based on rank rather than the relative rank is postponed later. 

Definition 1. Let k and n be positive integers. The relation R C is k- 
reducible, if there are colourings — >■ T) of with at most k colours 

(\Fi\ < k), for i = 0, ... ,n — 1, such that the following holds: Let — >■ 

Fo X •••F„_i,^(ao,... ,a„_i) = (co, . . . ,c„_i) where 



(We call ( the merge of £,i, i = 0, . . . ,n — 1. We use this notion even if the 
sequence is not complete.) Then for d,b € H", if £,{d) = ^(6) and d € R, then 
also b € R. R is reducible if it is k-reducible for some k € N* , otherwise R is 
irreducible. 

Since understanding the definition of irreducibility is absolutely crucial for 
the purpose of this paper, let me repeat the definition in a different form: A 
relation R is reducible if there is a finite alphabet E and a language L of words 
of length n such that, given a G A”, you can decide if a G i? by the following 
kind of a procedure. You first write down the tuple 



on the paper. Then, for each i = 0, . . . ,n — 1 consecutively, you first hide the 
component a^, then take a look at the remaining tuple 



zt (U{x) A -W{x) A {x = Xq\I X = xiM X = X2) 




Ci — (uq 5 ■ • ■ 5 1 ) ^2+1 5 ■ ■ ■ ; 1 ) • 



a — (uq, . . . , , . . . , On— 1) 
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write down your notes about the tuple, or actually just one letter Ci £ and 
then forget about the whole thing. After the process, you have written a word 

W — CqCi ■ * * Cn-1- 

If w € L, then a G R, otherwise a ^ R. 

Example 2. Let X be the set of all people and R the relation of all triples 
(a, b, c) £ such that a, h and c are sisters. Then R is reducible, even 2- 
reducible (actually all finite relations are reducible, so /c-reducible for some k, 
but the parameter k there may vary). Indeed, first hide c and take a look at 
the pair (a, b). If a and b are sisters, write down Y(es), otherwise N(o). Proceed 
similarly, and if in the end you have the word YYY on the sheat, a and b and c 
are sisters. 



The notion of irreducibility of n-ary R has been tailored equivalent to the 
statement r{R) = n. Let us then present a special case of Theorem |§] A mo- 
nadic quantifier of finite width Q is called a set quantifier if TZ{Q) is a set, or 
equivalently TZ{Q) C C” for some set C of cardinals and n £ N* . A monadic set 
quantifier Q of finite width is biassed towards infinite, if this C contains only 
infinite cardinals and possibly 0. Another way to put this is to say that there is 
an infinite k such that for every OJl £ Kq , the cardinality of is less than k and 
every element a of 2H has an infinite orbit under the automorhism group of 9Jl. 
Theorem |8] with the aid of Proposition [I] and the fact that irreducible relations 
are just those whose rank is the same as arity, implies the following theorem. 

Theorem 9. Let Q be a monadic set quantifier biassed to infinite with wd(Q) > 
2. Then the following are equivalent: 

1) Q is not definable in terms of monadic universe-independent quantifiers of 
width at most wd(Q). 

2) 1Z{Q) is irreducible. □ 

In the sequel, we need some technical results on irreducibility. 

Lemma 1. A non-unary relation R C is irreducible iff for every fc £ N* 
there is I G N such that R C {0, ... , Z — 1} C N" is k -irreducible. 



Proof. Let us prove the equivalence of negations. Suppose first R is reducible. 
By definition, R is then fc-reducible for some k G N* and for i = 0, . . . ,n — 1, 
there are colourings — >■ Tj such that for every d,b G N", if a £ i? and 

^(a) = ^(6), then b G R, where ^ is the merge of the colourings i = 0, . . . , n—1. 
Let Z £ N be arbitrary. Pick a new colour c* and put 






^i(a), for a £ {0, . . . ,Z - 1}" 
c*, otherwise. 



Then clearly colourings i = 0, . . . , n — 1 show that i? C {0, . . . , Z — 1} C N” 
is k-l-l-reducible. 
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To the other direction, suppose that there is k € N* such that for every 
I G N, the relation R C {0, ... ,1 — 1}" is A:-reducible. We apply techniques from 
m- Endowing the space P(N"') with its natural topology it can be shown that 
the set K {K = it!” — in the notation of ILTp of fc-reducible relations 

of N is a closed subset of P(N”). Now either for some I G N it holds that 
i? = i? n {0, . . . ,l — 1}" or R is an accumulation point of { i? fl {0, . . . ,1— 1}” | 
/ G N } C if. In both of the cases, R G K, i.e., R is fc-irreducible. □ 

Definition 2. Let R C N” be a non-unary irreducible relation. Then the com- 
plexity of irreducibility of R is in'. N* ^ N, 

inik) = min{ / £ N | i? fl {0, . . . ,l — 1}” C N” is k -irreducible. } 

4 Reverse Combinatorics: Proving van der Waerden’s 
Theorem 

We are finally in the position to choose the undefinability result V on which 
to base our analysis. For any ordinal a, denote ind(H„) = a. For each I G N*, 
let n = 2* and let Si be the monadic quantifier with the vocabulary r„ = 
{?7o) ■ • ■ ) Un-i} and with the defining class Ksi consisting of 911 £ Str(r„) such 
that all t/f^, 1 = 0 ,... , n — 1 are disjoint and infinite, but \Uf^\ < and 

n-2 

ind{\U^_,\) = J2^nd{\U!^\). 

i=0 

Let Vi be the following undefinability result (cf. [Lu, Example 4.6]). 

Vi : The quantifier Si is not definable by means 
of any universe-independent quantifiers of width 1. 

Moreover, let V be the statement: For every I G N, Vi holds. 

Let us analyze quantifiers Si in the spirit of the previous section. Let dJl G 
Ksi . If we enumerate the invariants in a suitable order, then 




where for 1 = 0, . . . , n — 1, is the number of elements in Uf^, but 

outside other predicates, zeros refer to the empty intersections, and A is the 
number of elements outside the predicates. Conversely, any 911 £ Str(r/) with an 
invariant of this form is in so that 

n{Sn) = {(H TUq ; ■ ■ ■ ; ^TTln — 1 1 ■ * ■ 5^7 A) I mo, . . . ,m„_i £ N, 

A yf 0 or m„_i yf 0, and m„_i = YhZq mj. 
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Constant components and vacuous variables (such as A) do not have any effect 
on the ranks of relations (cf. |LuJ 1 . nor does the replacement of by j. So for 
every I G N* and n = 2\ Vn is equivalent to the following statement: 

V'n'- The relation 



is irreducible. 

Where does the relevance of this result lie? Firstly, V gives an easy way to 
prove the monadic hierarchy theorem, i.e., the fact that extending the width 
has the capacity of enhancing the expressive power of monadic quantifiers. 
Since this motive comes from inside the field rather than from applications, 
let me point out that in the notation of the previous section, = Rq with 
q = (1, 1, , . . . ,1, —1) G Z", for every n G N, n > 2, so that this problematics 
is related to the proof that is not definable by monadic quantifiers. Unfor- 
tunately, irreducibility does not mean that the simple relations were useful 
for that particular proof. As it happens, r^(Tn) = 2 albeit r(T„) = n, which 
illustrates well the difference between finite and infinite arithmetics. In spite of 
this, we may view it as a mere pragmatic simplification that we use the result P 
rather than undefinability of as the basis of our analysis. Indeed, we could 
have introduced the notion of irreducibility for the relative rank, and then we 
would have got functions i® satisfying i®(n) > iji(n), for every n G N*, and we 
could have gone through an analysis in the similar vein for relations of form Rq. 
The trade-off would have been bad in the sense that we ought to have dealt with 
more complicated notions, with linear-algebraic technicalities (cf. [Lu, Section 
5]) and in the end, the results would have been almost the same as we shall get 
now. 

Abandoning this side-track, we first prove a technical lemma concerning par- 
tial functions. For tuples a and 6, let a~b be the concatenation of a and b. 

Lemma 2. Let k,n gN* and let f be an n-ary partial funetion on A whieh is 
fc" -I- 1-irredueible. Let : A" — >■ Fi, i = 0, . . . ,n — 1, be finite eolourings with 
at most k colours. Let ^ be the merge of ^i, i = 0, ... ,n — 1. Then there are 
d = (uq, . . . , ttn-i) G dom(f) and c G A such that c /(a) and ^(d "(/(a))) = 
^(dfc)), i.e., f{ao,... ,a„_i,/(a)) = ^(oo,... ,a„_i,c). 

Proof. Pick a new colour c* and put = {Fq x • • • x Fn-i) U {c*} and 



Then jUnj < fc” -|- 1. Let be the merge of i = 0, . . . , n. Since / is fc" -I- 1- 
irreducible, cannot carry all the information to determine if a tuple is in /, so 
there are d,b G A", c,d G A such that 1) ^*(a'(c)) = ^*(b~{d)) and 2) d = f(b) 
and 3) either d ^ dom(/) or c yf /(a). As 6 G dom(/), we have fn(b) = £,n{d), 



n— 2 





C(a'(/(a))): for a G dom(f) 



otherwise 
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whence the former possibility in case 3 is outruled. But $n(a) = ^n{b) says even 
more, namely that ^(a'(/(a))) = ^(b'' {f(b))). On the other hand, ^*(a'(c)) = 
Cib^id)) implies ?(a'(c)) = ^(b''{d)). Hence, $(a'(c)) = ^(a'(/(a))). □ 

Proposition 2. Let n G N with n > 2 and g: N” — >■ F &e a finite colouring. 
Then the irreducibility of T„+i implies that there are a G N" and d G Z \ {0} 
such that g{a) = g{a + dck), for fc = 0, . . . ,n — 1, where Ck is the unit vector 
having fc**' component 1. □ 

Proof. Pick a new colour c*. For each i = 0, .. . ,n — 1, colour 

dj — (do 1 • • • 1 ^i— 1 , ^i+ 1 5 ■ ■ ■ ; ^n) 

n—1 

by c* if 6 = d„ — X) dj < 0, otherwise colour a' by 

g(aQ, ■ • ■ , dj_i, bj dj+i, ■ • ■ , dyi_i). 

Let the resulting colouring be : N" — >■ FU {c*}. We can think of this colouring 
as an attempt to first recover the hole (i**' component) in the tuple and then to 
use colouring g. Now, by the previous lemma, the merge ^ of i = 0, . . . , n — 
1, does not carry all the information about T„. Consequently, there are a = 
(do, . . . , d„_i) G N” and d G Z, d yf 0, such that ^(d'(s)) = C(d"(s + d)) where 
s = Xr=o^®»- Unfolding the definition of the colourings, we see that for every 
i = 0, . . . , n — 1, 

p(d) ^z(^0? ' • • 1 1; ■ ■ ■ 5 ^n—li -s) 

= ^i(do,... ,dj_i,di+i,... ,d„_i, s + d) = g{a + dei). □ 



Now the goal is at hand: 

Theorem 10. (van der Waerden) Let y: N — >■ F &e a finite colouring and k G 
N. Then there are d, d G N such that d yf 0 and x(o) = y(® + ^d), for every 
i = 0, . . . , fc — 1. 

Proof. We may assume that fc > 2. Let n = fc — 1 G N* and consider the coloring 

n—1 

g: N” -)> F, g{xo,.. . ,x„_i) = xC^U + 

j=0 

The previous proposition implies that there are a = (do,... ,d„_i) G N" and 
d G Z \ {0} such that g{a) = g^a + def), for every z = 0, . . . , n — 1. Consequently, 
for a = Xy=o 0 + z = 0 ,... ,n — Iwe have 

X(d) = g{a) = g(d + def) = x(d + (z + l)d). 

Equivalently, for every i = 0, . . . , fc — 1, it holds that x(^) = x(® + '^d). If it 
happens that d < 0, the numbers d' = d + (fc — l)d and d' = — d > 0 fulfil the 
claim instead of a and d. □ 
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5 Fast-Growing Functions 

As it was mentioned in the introduction, the ideas of the previous section can 
be converted to results on combinatorial functions. One of the most interesting 
of such is the van der Waerden’s function Vh : N x N — >■ N. For some time 
there was speculation over whether W is even a primitive-recursive function 
until Saharon Shelah [Sj proved that W is actually quite low in the Grzegorczyk 
(or Ackermann) hierarchy. Still, it is possible that W 2 grows faster than the 
exponential tower, i.e., for every n G N, 

2 \ . 

N ^o' rn times 

W2{n) >2^ I 

The last step of the paper is to link complexities of irreducibilities to the di- 
scussion by proving an upper bound for W. It is worth mentioning here that 
a fast-growing complexity of irreducibility iji of i? means intuitively that the 
irreducibility of i? is difficult to discover, R is nearly reducible. 

Theorem 11. For every k,n gN, k,n>2, we have that 

W{n,k) < (n-l)iT„((fc+l)”-^ + l). 

Proof. Denote t = iT„((fc-|-l)" ^ + 1) and w = (n— l)t. Let %: {0, . . . ,w — 1} — >■ 
F be an arbitrary colouring with |F| < k. We need to show that {0, . . . , w — 1} 
includes a monochromatic progression of length k. For that purpose, we do the 
same tricks as in the previous section and consider the colouring g\ {0, . . . ,t — 

l}^FU{c*}, 



■ ■ ■ ; ^72—2) 



x(ELo^(^ + if defined 

c* , otherwise 



where c* is a new colour. For each i = 0, . . . ,n — 2 and 



a — (uq, • ■ • ; . . . , G M , 



let bi{a') = (ao, . . . , 0 ^- 1 , a„_i - 
recover from a, and let 



n-2 

X) -I- l)a;, . . . 



l—O^ l^i 



, an- 2 ) be the tuple we can 



^ 7 : { 0 ,... ,t-l}"-i^F’U{c*},^7(a) 



g{bi{a')), if defined 
c* otherwise 



Note that T' = T„ n {0, . . . , t — 1}" is a partial function from {0, ... ,t — 1}"“^ 
to {0, . . . ,t— 1}. By the choice of t, T' is {k + 1)”“^ -|- 1-irreducible, so applying 
lemma|2]to T' and the merge ^ of i = 0, . . . , n— 2, we find a = (oq, . . . , an- 2 ) G 
{0, . . . ,t— 1}”“^ and c G {0, . . . , t — 1} such that for s = 
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Cy^sKt and ^(a'(s)) = ^(a'(c)). Observe that u = ^ {n— l)s < 

(n — l)t = w. Hence, g(a) = x('^) ^^nd for every i = 0, . . . , n — 2, 

• 1 ^i—1^ , Q^n—2-} '^) 

— Q{^0: ■ ■ • ; ^i—li ^z+l? • ■ • j 2 ) 

= e{a) = x(u). 



On the other hand, x(®"('S)) = x('3^"(c)) implies that 

(^0 5 ■ • ■ 5 ^i— 1 ) ^i+1 5 ■ ■ ■ ; ^n— 2 -} c) 
Q (^ 0 j • ■ • ? 15 ^i+l 5 ■ ■ ■ 7 2) 

= g(d + dei) = g{d) 



where d = c — s yf 0. Hence x(^^) = x('^ + id), for every i = 0, . . . , n — 1, so 
that {0, . . . , w — 1} includes a monochromatic aritmetic progression of length n. 
Consequently, hb(n. A:) < w. □ 
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